Mobile Device And System For Automated Transport Mode Recognition And Corresponding Method Thereof

ABSTRACT

A method and system for automated transportation mode recognition based on sensory data measured by a plurality of sensors of a cellular mobile device of a user, the plurality of sensors at least comprising an accelerometer and a gyroscope, the plurality of sensors being connected to a monitoring mobile node application of the mobile device, wherein the mobile device measures time series of sensory parameter values based on measuring parameters obtained from the sensors, the measuring parameters comprise time series of sensory parameter values of a 3-axis accelerometer as sensor and time series of sensory parameter values of GPS-based speed measurements of a GPS receiver as sensor, and wherein the measured time series of sensory parameter values trigger the automated transportation mode recognition as input feature values to a gradient boosting machine-learning classifier, the transportation modes at least comprising the modes public transportation and/or motorcycle and/or cycling and/or train and/or tram and/or plane and/or car and/or skiing and/or boat, and the transportation mode recognition generating a transport mode label for a transport mode movement pattern of a trip.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of International PatentApplication No. PCT/EP2021/074817, filed Sep. 9, 2021, which claimspriority to PCT/EP2020/075548, filed Sep. 11, 2020, the contents of eachof which are hereby incorporated by reference in their entirety.

FIELD OF THE INVENTION

The present invention relates generally to transport mode recognitionsystems particularly to systems and methods for mobile phone sensorydata, as smartphone sensory data based transportation mode detection(TMR). More generally, the invention relates to mobile real-time systemsreacting dynamically to captured environmental or operationalparameters, in particular to automotive use monitoring, capturing, andreacting to automotive or user-related parameters in the context ofmotor vehicle operation. More particularly, the present inventionrelates to telematics based devices and systems for transport moderecognition. Finally, the invention also relates to telematics-basedreal-time recognition systems. The term telematics, in particulartraffic telematics, refers to sensory systems that are used forcommunications, instrumentation and control, and information technologyin the field of transportation. Thus, the present invention relates tothe use of telematics and/or mobile phone sensory data together withreal-time measuring, monitoring, dynamically and automatedly adaptingsystems based on captured and measured usage-based and/or user-basedtelematics data.

BACKGROUND OF THE INVENTION

With the technical advances in functionality and capability, mobilephone can act, nowadays, not only as a mere communication tool, butoften a monitor and service platform technically bridging applicationsand the end user. Modern mobile phones, such as smartphones, are beingintegrated with a variety of physical and virtual sensors capturing theuser's real-time contextual and situational information, such aslocation, activity, etc. The present invention is focused on mobilephone based transportation mode recognition systems and automatedmethods thereof. There are different systems in the prior art forautomated transport mode recognition (TMR), in particular in the contextof mobile devices, such as cellular mobile phones or smart phones. TMRsystems are known under different technical terms such as transport modedetection, travel mode detection, automated transport modeidentification, and recognition of travel pattern. The last categoryoften concerns machine learning, supervised learning, semi-supervisedlearning, unsupervised learning, statistical methods, and rule-basedmethod. The prior art systems often rely on GPS data, GPS tracks, GPSprocessing methods, trip identification, trip recognition, GPS datacleaning, GPS trajectories, and map matching. However, huge technicalhurdles remain in providing reliable and automated detection oftransportation modes of a smartphone user, inter alia, often related tothe limited availability of data processing power, the fragmentarinessor incompleteness of the sensory data of the mobile device, limitedbattery power etc.. Nevertheless, transport mode recognition isimportant for many technical applications, including transportationin-depth monitoring and machine-based intelligence analysis, urbanplanning, health monitoring, computer supported elder-care,epidemiology, etc. With the knowledge of travelers' transportation mode,targeted and customized electronic advertisements may be routed and sentto their devices. This information is also useful for the development ofcontext aware cell phones that sense the current context and adapt theirbehavior accordingly. Also, if the precise transportation modes ofindividual users are detected and/or monitored, it is possible toprovide a more realistic picture of travel demand. This knowledge mayhelp to determine the environmental impact of travel patterns, such ascarbon footprints of users and optimize the travel patterns or track thedaily step count of users and amount of calories they burn etc. Anotherapplication is the detection of real-time traffic states becausecompanies such as Google collect data from mobile phones in order toestimate and measure the traffic speed on roads, or technical trafficguidance systems or navigation systems for optimization of trafficlightening, in particular real-time steering. Further, in the context ofautomated detection of transport modes it is important to differentiatebetween the different applied classifications. The accurateclassification of a transport mode can be critical when the system isattempting to assist or enact measures to inform, warn, or protect auser or in measuring user-specific exposure parameters in the context ofpossibly impacting or affecting occurring event, as accidents ordisaster events.

However, as mentioned above, for GPS based recognition, poor receptionof GPS information and heavy energy consumption make real implementationdifficult; for GSM based recognition, it needs long time to observe thechange of cell information and the recognition accuracy is often low.Comparatively, merely accelerometer based recognition is often moreacceptable because: 1) the accelerometer consumes much less energy thanGPS; 2) the accelerometer needs a little starting time; 3) theaccelerometer can obtain sensor data all the time; 4) this method doesnot depend on any external equipment and will not affected by differentdeployment of cell towers. However, as to the accelerometer basedtransportation mode recognition on mobile phones, in real applications,it is difficult to determine the position and orientation of mobilephone because the user can place his mobile phone at any places he wouldlike to, such as in his hand, in the trouser pocket, in the bag, etc.Thus, some prior art systems try to recognize transportation modes usingthe combination of GPS and accelerometer. At there, the accelerationfeatures are extracted from the series of acceleration magnitude, whichis the value of acceleration synthetization of three axes. However, theaccelerometer signal averages over a reasonable time period often doesnot produce a good measure of the gravity related component. Then, thegravity vector estimate in turn cannot enable proper measurements of thevertical component and the magnitude of the horizontal component ofdynamic acceleration caused by user. Even if two methods to recognizeuser's physical activities are used, unseen the problem of energyconsumption of the mobile device, such acceleration decomposition basedapproaches do not perform substantially better than mere accelerationsynthetization based approaches.

In any case, to determine the mode of transportation based onsmartphones, the data from different built-in smartphone sensors istypically used. Most modern smartphone devices have sensors that measuremotion, orientation, and various environmental conditions. They arecapable of providing data with high measuring precision and accuracy.These sensors are useful for monitoring three-dimensional devicemovement or positioning, or for monitoring changes in the ambientenvironment near a device. Motion sensors include accelerometers,gravity sensors, gyroscopes, and rotational vector sensors. Positionsensors include orientation sensors and magnetometers. Environmentalsensors include barometers, photometers, and thermometers. In additionto mobile device sensor information, some external data source can alsobe valuable.

The smartphone sensors and external data sources typically employed intransportation measuring can be summarized as follows: (A)Accelerometers are able to measure the physical motion of a solidobject. Id est, they measure the acceleration force that is applied to adevice on all three physical axes, including the force of gravity.Accelerometers are primarily used for orientation sensing insmartphones. However, transportation measuring shows that theacceleration generated during human movement varies across the body anddepends upon the activity being performed. The key feature that makesthis sensor attractive is low energy consumption; (B) Gyroscope measuresa device's rate of rotation around each of the three physical axes. Itcan provide orientation information, and provides an additionaldimension to the information supplied by the accelerometer. Gyroscopesare typically characterized by low power consumption, but are, however,prone to error accumulation as a result of significant calibrationerrors, electronic noise, and temperature; (C) Magnetometer measures theambient geomagnetic field for all three physical axes. It providesmobile phones with a simple orientation in relation to the Earth'smagnetic field; (D) Global Positioning System (GPS) sensor provides theposition and velocity of the user that is measured based upon thedistance of the mobile phone and each of a number of satellites in twodimensions. Connection to at least three satellites is required fortwo-dimensional positioning, and the precision increases with morevisible satellites. GPS does not work indoors, and is thereforeprimarily used for outdoor positioning. A further technical limitationis, that it is characterized by reduced precision of positioning indense urban environments, due to the fact that buildings reflect andocclude satellite signals. GPS is considered as the most power consuminglocalization technique for mobile computing, and it reduces the batterylife of the phone significantly. The accuracy of this system is between50 to 80 meters and can be improved to an accuracy of up to 10 meters;(E) Cellular network signals are used by the phone for calls and datatransfer. The most widespread cellular telephony standard in the worldis Global System for Mobile Communication (GSM). A GSM base station istypically equipped with a number of directional antennas that definesectors of coverage or cells. A cell is therefore a geographic region ina cellular communication network within which mobile devices cancommunicate with a particular base station. Each cell has a unique cellidentifier. The fluctuation pattern of cell identifiers together withsignal strength can provide information on the position of a phone. Tocollect this type of data, an application that measures and records thesurrounding radio environment has to be installed on a mobile device.Mobile phones can be tracked in outdoor and indoor contexts. A precisionvaries depending on cell size from 50 to 200 meters, but can deteriorateeven more in low density areas. Cellular network signals are associatedwith “ping-pong” phenomenon, which appears when a user is within thecoverage of two or more stations. Signal strength from the stationsfluctuates and causes repetitive changes of associated cell even whenusers are stationary. The data from mobile phone operators can beanalyzed consisting of anonymous location measurements generated eachtime a device connects to the cellular network (e.g. when a call isplaced or received, when a short message is sent or received, when theuser connects to the Internet, etc.). However, these measurements areavailable only during the time that the device is in use, or when theassociated cell changes over time (e.g. during a trip); (F) Bluetoothallows wireless connectivity and short range communication. Bluetoothsensors are able to sense devices in their vicinity, and to obtain theirBluetooth identifiers, names and types. The range of Bluetooth scannersand penetration rate vary between 10 to 100 meters, respectively between7% and 11%; (G) WIFI provides wireless connectivity to devices inside aWireless Local Area Network (WLAN). The WLAN provides communicationranges of up to 100 meters and allows to track devices outdoor andindoor. Smartphones do not need to be logged on to the WLAN, but theirWIFI antennas has to be turned on. The positioning accuracy is low. Itis possible to improve the localization in case when there is more thanone access point available using for instance signal triangulation andfingerprinting. WIFI is the most power-demanding sensor after GPS whenused to provide location information. The effect called “ping-pong” isalso typical for WIFI data; (H) Other sensors include barometers thatmeasure atmospheric pressure and can be used to detect how high thephone is above sea level, thermometers and humidity sensors that measureambient temperature and air humidity, cameras, microphones, etc.; and(I) External data sources can provide additional useful information intransportation measuring. They include network infrastructure data androute maps, as well as the time schedules of public transportation modesin a static or a real-time form.

In the prior art systems, raw data measured and collected by differentsmartphone sensors are typically transformed into more computationallyefficient and lower dimensional sets of features. The extracted featuresare intended to be informative and e.g. relevant for the learning task.A variety of feature-extraction techniques are used in the state of theart, based on different data processing approaches, algorithm structuresand statistical procedures. The raw sensor data are typically segmentedinto several windows and features are extracted from a window ofsamples. The window size, as well as the sampling frequency, areimportant parameters, as they both affect computation and powerconsumption of sensing algorithms. Smaller window sizes causeclassification accuracy to suffer due to certain features not beingeffective (e.g. accelerometer frequencies) and larger window sizes mayintroduce noise in the data.

Time domain and frequency domain features are used for transportationmode detection tasks. Time domain features are used to characterize theinformation within the time varying signal. Many prior art systems useraw speed or acceleration data, and GPS positioning information overtime as input features. The difference in distance covered betweenmeasurements and heading changes are used in addition. For accelerometersignals, the features such as mean, standard deviation, median, minimum,or maximum of the signal are the most commonly used in time domain. GSMsignal strength and cell tower fluctuations are utilized for inferringdifferent states of user motion. Frequency domain features are regardedas technically more computationally demanding compared to the timedomain features. This is due to an additional processing step, relatedto the data transformation from the time to the frequency domain. Anexample of these features is the peak frequency of the power spectraldensity of the accelerometer signal. Finally, features extracted basedon external data typically include bus location closeness, bus stopcloseness and rail line closeness.

The prior art algorithm structures used for transportation modedetection can typically be categorized as discriminative or generative.Generative algorithms are based on modelling and/or simulatingclass-conditional probability density functions and backward in timeprobabilities. As such, they allow to generate samples from the derivedjoint distributions, and are typically flexible in expressingdependencies in complex learning tasks. For this group, structures ofprior art systems comprise the algorithms Naïve Bayes, BayesianNetworks, Mixture Models and Hidden Markov Models. Discriminativealgorithms do not attempt to model underlying probability distributions.Instead, they are focused on a direct estimation of posteriorprobabilities. Popular discriminative algorithms include Support VectorMachines, Neural Networks, Nearest Neighbor, Decision Tree, RandomForests, Clustering, etc.

One further deficiency of the prior art system lies in the fact that theprior art systems in the field use a great variety of differentcategorization of transportation modes. In general, transportation modescan be roughly classified into motorized and non-motorized (alsoreferred as soft modes). Motorized modes include cars, motorcycles,trucks, buses, trams, metros, and trains. Walking, running, and bikingare typical representatives of soft modes. Soft modes can e.g. beconsidered to contribute to the reduction of congestion and pollution,the enrichment of the local environment, the improvement of quality oflife, enhanced accessibility, and social equity. An often foundgovernmental objective in many countries is therefore the promotion ofsoft modes, which together with public transport represents anenergy-efficient and low resource-consuming means of transport, inparticular in order to achieve the climate goals as set in the Parisclimate conference of 2015. It seems to be clear that differenttransportation modes should show different mobility patterns. Forinstance, motorized modes generally have a higher speed than soft modes.FIG. 1 shows examples of prior art measurements of normalized histogramsof the recorded speed data and the estimated speed distributions forfive modes. Technically, walking, and biking modes are typically easierto differentiated from other modes. However, slow walking can be rathersimilar to no movement, and biking can be very similar to the cars. Asfor the latter, the similarities are reflected through similar speedsand routes of bikes and cars in the cities and absence of fixedschedules. Buses are supposed to follow fixed routes and schedules,which places them apart from the categories above. However, due to highvolumes of traffic, significant discrepancies may occur making buses oneof the categories that are more difficult to measure and predict. Trams,metros, and trains follow routes and schedules like buses, but are notsubject of delays induced by traffic congestion. Their routes maycontain underground parts, causing poor quality or non-existence ofcertain smartphone signals. Some of the prior art systems additionallyconsider stationary mode, which refers to a not moving state of anobject, and as such significantly differs from other categories.Whatsoever, there is strong demand in the technical field for a reliablesystem for efficient automated transport mode recognition.

In the following, some of the prior art systems are discussed moredetailed: (i) One of the prior art systems is based on an unsupervisedmethod of learning a Bayesian model from a measured GPS sensor stream.This technical approach simultaneously learns a unified model of thetraveler's current mode of transportation and her most likely route.Car, bus, and walking are considered transportation mode categories. Forthis kind of prior art systems it can be shown that the accuracy of theapproach is improved by adding more external GIS knowledge; (ii) Anotherprior art system is based on using the patterns of GSM signal strengthfluctuations and changes to the current serving cell and monitoredneighboring cells it is possible to distinguish between various statesof movement such as walking, driving in a motor car, and remainingstationary. The system uses a Hidden Markov Model for inferring thecurrent activity of the cell phone carrier; (iii) In a further prior artsystem, coarse-grained GSM data from mobile phones are used to recognizehigh-level properties of user mobility. This approach is a two-stageclassification scheme. The first stage classifies an instance asstationary or not. If the instance was classified as not stationary, asecond classifier determines if the instance was walking or driving.Both classifiers are trained using a boosted logistic regressiontechnique. The algorithm structure is able to distinguish if a person iswalking, driving, or remaining at one place. Typically, such two-stageclassification have a superiority compared to Naïve Bayes, SupportVector Machines, and heuristic-based systems; (iv) Another prior artsystem is focused on the transportation mode of an individual based onGPS and accelerometer smartphone data. The goal in such systems is todetermine whether an individual is stationary, walking, running, biking,or uses motorized transport. This classification system consists of adecision tree followed by a first-order Hidden Markov Model; (v) Oneprior art system uses a Decision Tree-based algorithm, that utilizes GSMand WIFI traces for transportation mode detection. The algorithmstructure infers a user's mobility, being either dwelling, walking, ordriving; (vi) A further prior art system is based on supervised learningto infer transportation modes from GPS logs. The system uses agraph-based post-processing algorithm, that considers both theconstraint of real world and typical user behavior based on location ina probabilistic manner. This approach focuses on transportation modesincluding driving, walking, taking a bus and riding a bicycle; (vii) Oneprior art system is based on a rule learning algorithm to determinewhether a smartphone user is sitting, standing, walking, or running. Thesystem is based on accelerometer measuring data. The derived classifiersexecute in part on the phones and in part on the backend servers toachieve scalable inference; (viii) A further prior art system is aclassification system that uses a mobile phone with a built-in GPSreceiver and an accelerometer. The transportation modes identifiedinclude whether an individual is stationary, walking, running, biking,or in motorized transport. The classification is composed of a decisiontree followed by a discrete Hidden Markov Model; (ix) A prior art systemuses a technical approach which is based on inferring a mode oftransportation based on the GPS data collected via smartphones and theunderlying transportation network data. The system is able to detectvarious transportation modes including car, bus, train, walking, bikingand stationary. Five different inference models, Bayesian Net, DecisionTree, Random Forest, Naïve Bayesian and Multilayer Perception, cantypically be used with this system; (x) Another prior art transportationmode detection system uses speed statistics derived from GPS andcellular network information and measurements, together with statisticsobtained from accelerometer samples. The system uses decision tree rulesand is able to distinguish between bus, Mass Rapid Transit (MRT) andtaxi; (xi) A prior art system infers multi-modal itineraries from acombination of smartphone sensor data (e.g., GPS, WIFI, accelerometer)and the transport network infrastructure data (e.g., road and rail maps,public transportation timetables). In the first phase, the system uses adynamic Bayesian network based on network and sensor data, and candistinguish between and recognize walking, biking, road vehicle, andtrain. The second phase attempts to match parts recognized as roadvehicle or train with possible bus, train, metro, or tram based on theirtimetables; (xii) A probabilistic prior art system is based on inferringthe transport modes and the physical paths of trips. This system usesdata from GPS, Bluetooth, and accelerometer smartphone sensors. Thesystem is based on a smartphone measurement model that is able togenerate the likelihood of observing the smartphone data in themulti-modal transport network. It is formed of a structural travel modelthat captures the dynamic of the state of a smartphone user in thetransport network, and sensor measurement models that capture theoperation of sensors. The system distinguishes between five transportmodes: walking, biking, car, bus and metro; and finally (xiii) a furtherprior art system is based on accurate determination of thetransportation mode while minimizing the strain on the phone's processorand battery. The data from the internal sensors such as accelerometer,gyroscope and magnetometer are used. Different algorithm structures canbe used, such as Decision Trees, Random Forest and k-Nearest Neighbors,however, typically Decision Trees structures are used. In this system,the transportation modes are limited to walking, running, riding a bikeand driving a car.

In summary, the prior art systems are typically based on using one ortwo sensors, mostly GPS and accelerometer for transportation modedetection, mainly due to energy-consumption and computing-power reasons.The accelerometer is particularly attractive, given that it capturesrelevant features while being energy efficient. Thus, prior art systemsthat use three or more smartphone sensors are quite rare, and theaccuracy and reliability of the detections is relatively low. Ingeneral, accuracy of transportation mode detection is typically higherif more data sources are utilized. External data sources are rarelyemployed in prior art systems, and typically include transportationnetwork data and timetables of public transportation services. Thefeatures used in the classification task typically depend on the usedsensors and the classification techniques. Most of the prior art systemstry to use as few features as possible in order to minimize thecomputational burden of feature extractions as well as the risk of modelover-fitting. However, this goes often to the disadvantage of accuracyand reliability of the systems. Due to computational consumptionreasons, generative model structures are often used in cases when mobilephones are used only as a sensing system and the data analysis andclassification are performed on back end servers. If the detection isintended to run on mobile devices directly, the use of generative modelstructures is technically limited due to their computational costs incontrast discriminative models. For such cases, technical approachesbased on Decision Trees are typically used to achieving satisfactoryaccuracy while using the least resources.

All prior art systems are focused on stationary, walking, biking andmotorized modes. In general, walking and stationary modes are predictedwith higher accuracy compared to other modes by the prior art systems.This is a fundamental disadvantage of the known prior art systems.Contributions that achieve high accuracies are typically forced to belimited to the detection of a unique motorized transport mode. This is atechnical problem given that a key challenge in transportation modedetection appears to be the differentiation between motorized classessuch as bus, car, train and metro. With multiple motorized modes, thedetection problem becomes technically difficult using the known priorart systems due to common characteristics that these modes share (e.g.the average speed and accelerations). Some prior art systems that employexternal data have demonstrated their added value in detecting variousmotorized transportation modes. However, in the prior art, publictransportation modes can only be detected when external data sources arecombined with smartphone sensor data. Sometimes, external data are alsoused to improve the accuracy of detecting running and biking as well,which can be similar to driving. However, there it is less efficient. Afurther technical problem of the prior art systems is that GPS sensorsmay not provide reliable location and time information. The location ormovement, for example, may be recorded via GPS devices or GPS-enablecellular phones or smart phone. As described above, Global PositioningSystem (GPS) is a satellite based navigation system that providespositioning, navigation, and timing services to users on a continuousbasis. GPS is made up of three parts: orbiting satellites, control andmonitoring stations, and GPS receivers. A GPS need to receiver obtainssignals from at least two GPS satellites to measure thethree-dimensional location (latitude, longitude, and altitude) plus thetime of a GPS-enabled device. However, there may be a lack of trackinginformation attributed to a poor or a nonexistent connection to GPSsatellites. This poor or non-existent connection to GPS satellites maybe due to the user being inside a building or other structure, due toreflection off the exteriors of large buildings or other objects, due todestructive interference of the signals from towers in urban areas, ordue to the type of construction materials used in some buildings. Thus,it is difficult to rely on the GPS sensor essentially or alone to tracklocations of the user and detect transportation modes.

Finally, in the prior art, EP 3 091 498 A1 discloses amobile-device-based system for classifying a mode of transportationduring a trip. The system comprises a mobile device including a locationdetection system, as a GPS system and an accelerometer. Location dataand acceleration data are collected during the trip by the mobiledevice. The system further receives contextual data related to aplurality of transportation systems and processes the location data andfirst contextual data using a first transportation mode classifierassociated with a first set of transportation systems and secondcontextual data using a second transportation mode classifier associatedwith a second set of transportation systems. The mode of transportationis classified by data processing during the trip and update theclassifiers based on user input. Further, U.S. Pat. No. 10,630,723 B1discloses a method for automated adjustment of policy characteristicsbased on a determined similarity between routes. A similarity metric isdetermined indicating the similarity between a first route followed by avehicle and/or driver and a second (e.g., previous) route followed bythe vehicle and/or driver. A similarity metric indicates the similarityin movements, and changes in movement, exhibited by the vehicle on theroutes. The similarity metric is applied through analysis of real timedata collected by in-vehicle sensors, mobile user devices, externalsensors or other data sources. Based on the similarity metric, apremium, a deductible, a price, or other parameters of a policy aredetermined. The policy characteristics can be adjusted (e.g., in realtime) based on the analysis according to changing risk conditions if adriver is following routes that are dissimilar from typical routes.

SUMMARY OF THE INVENTION

It is one object of the present invention to provide a more reliable andaccurate automated system for transportation mode recognition of a userbased on measured sensory data of a mobile device, such as a cellularmobile phone or a smart phone. The mobile device should be able toautomatically detect and identify a user activity, and distinguishingbetween different walking and driving modes, or other manual andmechanical locomotion patterns. Independent of possibly available,dedicated in-vehicle hardware, maybe providing alternative solutions,the invention shall provide an appropriate system and method fortransportation mode recognition, in particular real-time transportationmode recognition, and more particularly continuous real-time trackingand transportation mode recognition, solely based on the availablesensors of a smartphone user. The output signaling of the transport moderecognition should achieve an accuracy to be usable for as inputsignaling for electronically triggering or steering various technical,transport-mode dependent processes. For example, the present automatedtransport mode recognition should be applicable to location and/ortransport mode dependent, automated systems, such as personal navigationsystems enabling to automatically switch modes to appropriate datacapturing processes or formats, the mode reflecting the type oflocomotion. Other location and/or transport mode dependent, automatedsystems can e.g. comprise usage-based risk-transfer processes and/orappropriate usage-based real-time risk-transfer systems, the automatedusage-based real-time risk-transfer systems being enabled by thesignaling of the automated transport mode recognition to perform thelocation and/or transport-mode dependent risk-transfer processes.

According to the present invention, these objects are in particularachieved with the features of the independent claims. In addition,further advantageous embodiments can be derived from the dependentclaims and the related descriptions.

According to the present invention, the above-mentioned objects forautomated transportation mode recognition based on sensory data measuredby a plurality of sensors of a cellular mobile device of a user are inparticular achieved in that, the plurality of sensors at least comprisean accelerometer and a GPS sensor and the mobile device comprises one ormore wireless connections, wherein by at least one of the wirelessconnection, the cellular mobile device acts as a wireless node within acellular data transmission network by means of antenna connections ofthe cellular mobile device to the cellular data transmission network,and the plurality of sensors being connected to a monitoring mobile nodeapplication of the mobile device, wherein the monitoring mobile nodeapplication captures usage-based and/or user-based sensory data of thecellular mobile device and/or the user of the cellular mobile device, inthat the mobile device measures time series of sensory parameter valuesbased on measuring parameters obtained from the mobile device's sensors,the measuring parameters comprise time series of sensory parametervalues of a 3-axis accelerometer as sensor and time series of sensoryparameter values of GPS-based speed measurements of a GPS receiver assensor, wherein the GPS sensor measures the mobile device's longitude,latitude and altitude position by measuring different speed of lightdelays in the signals coming from two or more satellites, and in thatthe measured time series of sensory parameter values trigger theautomated transportation mode recognition as input feature values to agradient boosting machine-learning classifier, the transportation modesat least comprising the modes public transportation (train and/or tramand/or bus) and/or motorcycle and/or cycling and/or plane and/or carand/or skiing and/or boat, and the transportation mode recognitiongenerating a transport mode label for a transport mode movement patternof a trip.

As an embodiment variant, a supervised learning structure can e.g.applied to the gradient boosting machine-learning classifier during asupervised learning phase, wherein transport mode movement patterns ofmeasured trips are stored to a trips database, in that the mobile devicemeasures sensory movement parameters based on measuring parametersobtained from mobile devices' sensors of a heterogeneous set of users,wherein transport mode movement patterns of a trip are identified fromthe measured sensory movement parameter values, and wherein each tripcomprises measured sensory movement parameter values of GPS positions bya GPS-sensor device, and of acceleration forces being applied to themobile device on all three physical axes by a 3-axis accelerometer, andof operating system activities parameter values of an operating systemof the mobile device, and a transport mode label value, and in thattrips with transport mode labels detected by the gradient boostingmachine-learning classifier are fed into a user back-loop for dynamiccorrection by a user associated with the respective trip and saved tothe trips database by updating the learning transport mode movementpatterns of measured trips in the trips database.

As further embodiment variants the trips database can e.g. be monitoredby the system automatically detecting changes in the trips database,wherein upon detecting a change in the trips database, the supervisedlearning phase is reinitiated. The trips database can e.g. be updatedcontinuously and/or dynamically based on the applied user back-loop. Forthe supervised learning phase, the trips of the trips database can e.g.be preprocessed by filtering out transport mode movement patterns, whichhave a time duration shorter than 1 minute and/or comprise less than 30GPS positions and/or do not have a proper transport mode labelling. Forthe supervised learning phase, the trips of the trips database can e.g.be preprocessed by filtering out transport mode movement patterns havingduplicated GPS locations by timestamp and/or transport mode movementpatterns with GPS locations having negative speed and/or transport modemovement patterns having GPS locations with an accuracy>50 m. Upondetection by the gradient boosting machine-learning classifier and thegeneration of the transport mode label for a transport mode movementpattern of a trip, the trips can e.g. be postprocessed by a set of hardcoded rules, reinforcing avoidance of incorrect and/or insufficientlyconfident recognition. User's routines can e.g. be automaticallydetected by a trip familiarity-based recognition structure increasingthe accuracy of the automated transportation mode recognition. Thetransport mode movement patterns of measured trips stored to the tripsdatabase can e.g. be processed for data enrichment, where the dataenrichment process comprises route-matching of the trips with thetransport mode movement patterns based on roadmaps and/or GIS-geometrymapping of the trips with the transport mode movement patterns based onspatial and geographic GIS data, and/or public transport mapping basedon public transport road maps and timetable data. The measuringparameters of the time series of sensory parameter values of theGPS-based speed measurements can e.g. further comprise extracted averageGPS-based speed measurements and/or standard deviation of the GPS-basedspeed measurements and/or percentiles values from 0 to 100 of theGPS-based speed measurements in predefined percentile steps. Thepercentile steps can e.g. comprise a granularity of a factor 10. Themeasuring parameters of the time series of sensory parameter values ofthe GPS-based altitude position measurements can e.g. further comprisemeasuring a standard deviation extracted from the altitude positionmeasurement. The measured time series of sensory parameter values cane.g. comprise GPS-based acceleration measurements derived based on themeasured ratio between (a) the measured speed difference between ameasured set of GPS parameter values and a measured subsequent set ofGPS parameter values, and (b) the measured time difference between themeasured set of GPS parameter values and the measured subsequent set ofGPS parameter values. The GPS-based measurement of the acceleration cane.g. comprise measuring a standard deviation extracted from theacceleration measurement and/or measuring a variance value of themeasured GPS-based accelerations by measuring the angle between tripletsof consecutive GPS points. The measuring parameters of the time seriesof sensory parameter values of the accelerometer measurements can e.g.further comprise percentiles values from 0 to 100 of the accelerometermeasurements in predefined percentile steps and/or interquartile rangevalues measured by the difference between the 75^(th) and the 25^(th)percentile. Said percentile steps can e.g. comprise a granularity of afactor 10. In case of two or more accelerometer measurements having thesame timestamps, the last one with respect to an accelerometermeasurement order can e.g. be selected, and an accelerometer measurementnorm value is generated over the accelerometer measurements of themeasured trip, wherein an average of the accelerometer measurements ofthe measured trip is removed form said measured trip. The operatingsystem activities parameter values of an operating system of the mobiledevice can e.g. comprise unique timestamp and a map of labels withprobability measures, wherein in case of the absence of a label, theprobability measure is set to 0. Said labels of the map can e.g. benormalized to ‘Automotive’, ‘Cycling’, ‘OnFoot’, ‘Running’,‘Stationary’, ‘Unknown’, ‘Walking’, and ‘Tilting’ denoting a featurevector for naming compliance between the two operating systems. Forachieving a forward integral generation by the system, a labelprobability can e.g. be assumed to be valid until a next event ismeasured, wherein each label probability is multiplied by the measuredmilliseconds elapsed until the next event, or until the end of the tripfor the last received activity event, wherein this multiplication isperformed for each label of the label list, wherein the multiplied labelprobabilities for each label are summed up, and wherein each sum isdivided by the difference between trip end time and the first activityevent time, both in milliseconds. In case of a label being neverreturned, the corresponding feature can e.g. be set to 0, so that ifthere are no activities at all for a measured trip, all the operatingsystem activities parameter values are set to 0. The data enrichmentprocess comprising public transport mapping based on public transportroad maps and timetable data can e.g. comprise identifying for a set ofmeasured GPS location parameters candidate stops as sequences of pointsthat fulfill the condition of: (i) measured speed<=3 m/s; and (ii)candidate sequences are longer than 5 seconds, wherein theidentification is performed after applying a moving average with windowlength 9 over an array of measured speeds, replacing each sample theaverage of the sample itself and the 4 samples before and after, andwherein for each of these candidate sequences, the average latitude andaverage longitude is generated, obtaining a candidate stop position foreach sequence/stop. Further a hyperparameter configuration can e.g. beapplied to the gradient boosting machine-learning classifier comprisingthe values 225 for the n-estimators, 0.03 for the learning-rate, 30 forthe max-depth, 50 for the num-leaves, 0.8 for the subsample, 0.7 for thecolsample-bytree and 5 for the min-sum-hessian-in-leaf.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be explained in more detail, by way ofexample, with reference to the drawings in which:

FIG. 1 shows a block diagram schematically illustrating an exemplarysystem for the inventive transport mode recognition system 113. Theautomated transportation mode recognition 113 is based on sensory data3/31/32/33 measured by a plurality of sensors 102 of a cellular mobiledevice 10 of a user 6. The plurality of sensors 102 at least comprise anaccelerometer 1025 and a GPS sensing device 1024. The mobile device 10comprising one or more wireless connections 105. By at least one of thewireless connection 105 the cellular mobile device 10 acts as a wirelessnode 221, . . . , 225 within a cellular data transmission network 2 bymeans of antenna connections of the cellular mobile device 10 to thecellular data transmission network 2. The plurality of sensors 102 isconnected to a monitoring mobile node application 101 of the mobiledevice 10. The monitoring mobile node application 101 capturesusage-based 3 and/or user-based sensory data 3 by means of the sensors102 of the cellular mobile device 10 and/or the user 6 of the cellularmobile device 10. As an embodiment variant, the mobile device 10 canalso access sensory data of external sensory devices, as e.g. in-carsensors, or smart-house sensors, over interfaces as Bluetooth or WIFIetc.

FIG. 2 shows a block diagram schematically illustrating an embodimentvariant of an exemplary system for automated transportation moderecognition 113. A supervised learning structure 1136 is applied to thegradient boosting machine-learning classifier 1131 of the transportationmode recognition 113 during a supervised learning phase. Transport modemovement patterns 11351 of measured trips 1135 are stored to a tripsdatabase 33. The mobile device 10 measures sensory movement parameters311,312,313;321,322,323 based on measuring parameters obtained fromsensors 102 of mobile devices 10 of a heterogeneous set of users 6.Transport mode movement patterns 11351 of a trip 1135 are identifiedfrom the measured sensory movement parameter values311,312,313;321,322,323 by the transportation mode recognition devicesor system 113, wherein each trip comprises at least measured sensorymovement parameter values 311,312,313;321,322,323 of GPS positions bythe GPS-sensor 1024/102, and of acceleration forces being applied to themobile device 10 on all three physical axes by a 3-axis accelerometer1025/1902, and of operating system activities parameter values of anoperating system of the mobile device 10, and a transport mode labelvalue 1134.bTrips (1135) with transport mode labels 1134 detected by thegradient boosting machine-learning classifier 1131 are fed into a userback-loop 1136 for dynamic correction by a user associated with therespective trip 1135 and saved to the trips database 33 by updating thelearning transport mode movement patterns of measured trips 1135 in thetrips database 33. The reliability of the automated transport-moderecognition increases as more data points are accumulated. If the system113 fails to recognize the mode of transport correctly, users have theoption to manually correct the predicted transport mode in the system113. The changes are automatically detected and the supervised learningstructure is retrained in order to avoid repetition of the same mistakeand improve the overall performance: the TMR-system's 113 predictioncapabilities improve in a continuous cycle.

FIG. 3 shows another block diagram schematically illustrating exemplaryhow for each of candidate sequences, the average latitude and averagelongitude is generated by the TMR system 113, obtaining a candidate stopposition for each sequence/stop. By using the public transport algorithminputs and outputs, additional features can be generated by the TMRsystem 113: (i) the number of candidate stops of the trip (trajectorystops) (CandidateStopsCount), (ii) the number of candidate stops of thetrip (trajectory stops) divided by the cumulated sum of haversinedistances between the 16 sampled GPS points, ordered increasingly bytime, in meters (CandidateStopsCountNormalized), (iii) the number ofsuggested stops for the best matching API suggestion(PublicRoutingNumStops), (iv) the cumulated haversine distance of thesuggestion stops, in order of traversal, divided by the cumulatedhaversine distance of the 16 sampled GPS points(PublicRoutingDistRatio), (v) the cumulated haversine distance of thecandidate stops, divided by the cumulated haversine distance of the 16sampled GPS points (PublicRoutingCandidateDistRatio), and/or (vi) thepercentiles from 0 to 100, with step 10, of the minimum distances fromthe suggestion stops to the candidate stops (this is the standard publicstop algorithm). These features can be generated for all thesuggestions, but the ones selected are the ones regarding the suggestionwith minimum distance between suggestion stops and candidate stops.

FIG. 4 shows block diagram schematically illustrating an exemplaryperformances achieved by the overall automated TMR system 113, which aredescribed by the confusion matrix of FIG. 4 and the following table, andobtained through a 5-fold Cross-Measurement with a leave k-users outsplitting technique, in order to reduce overfitting.

Transport Mode Support Recall Precision boat 12 100.00% 100.00% car12710  98.68%  94.98% cycling 407  71.74%  91.54% motorcycle 851  53.94% 88.78% other 13  30.77%   2.60% plane 115  77.39%  88.12% public 1000 77.90%  92.63% skiing 349  93.70%  92.90% train 316  82.59%  95.96%

FIG. 5 shows a diagram illustrating an exemplary architecture of thedata preprocessing. Before being inputted to the machine learningstructure of the TMR system 113, the time series pass through thefollowing preprocessing steps: (i) Rotation of the 3-axis accelerometerfrom the smartphone reference system to the vehicle reference system,(ii) Alignment between accelerometer and GPS, sharing a common 10 Hzsampling grid, and (iii) Each trip is split into multiple 5 minutes longmini-trips. The final input to the TMR system 113 is for this exemplarycase a 4-dimensional time series, with a fixed length of 3000 timesteps(5 minutes*10 Hz).

FIG. 6 shows a diagram illustrating an exemplary performance of theautomated TMR system 113. Performances have been measured and assessedthrough a 5-fold Cross-Measurement or Cross-Validation with a leavek-users out splitting technique, leading to the results shown in FIG. 6and the following table:

Transport Mode Precision Recall F1−Score Car 93.63% 94.58% 94.10% Moto89.02% 87.23% 88.11%

FIG. 7 shows another block diagram illustrating schematic an exemplaryoverview of the architecture of the Transport Mode Recognition systempart of system 1.

FIG. 8 shows a block diagram illustrating schematic an exemplaryoverview of the trip extraction process.

FIG. 9 shows a diagram illustrating schematic an exemplary Car/NoCarperformance (F1 score) with minimum accuracy as a free parameter.

FIGS. 10 a and 10 b show diagrams illustrating schematic an exemplaryTMR performance (F1 score) with number of sampled points as a freeparameter.

FIG. 11 show a diagram illustrating schematic an exemplary candidatestops extraction.

FIG. 12 shows a block diagram illustrating schematic an exemplary tripenrichment process.

FIG. 13 shows a block diagram illustrating schematic exemplary featureextraction modules.

FIG. 14 shows a diagram illustrating schematic an exemplary recursivefeature elimination used for the transport mode recognition TMR of thesystem 1.

FIG. 15 shows a diagram illustrating schematic an exemplary Gridexploration of the number of trees and tree depth parameters.

FIG. 16 shows a diagram illustrating schematic an exemplary early TMRdetection for car/nocar classifier, F1 score.

FIG. 17 and FIG. 18 show respectively the performances of the currentTMR service and the proposed solution, where FIG. 17 shows thedistributions of true labels, performance of the deployed solution(baseline), and FIG. 18 shows the distributions of true labels, proposedsolution.

FIG. 19 shows a diagram illustrating schematic an exemplary F1 scorevarying TMR label weight (probability mass assigned to the automaticlabel).

FIG. 20 shows a diagram illustrating schematic an exemplary the designof an index value used for the generation of the familiarity andfamiliarity score, so that it orders the users with the following order,given the clusters dimensions (x-axis: cluster number, y-axis: clusterdimension).

FIG. 21 shows a diagram illustrating schematic the correlation betweenthe Gini index and the index (denoted as “new index”), used in theproposed embodiment variant.

FIG. 22 shows a diagram illustrating schematic an exemplary user goingfrom the same point A to the same point B, but passing through differentlinks. This behavior causes low aggregation in Link familiarityembodiment variant and high aggregation in Start Stop embodimentvariant.

FIG. 23 shows a diagram illustrating schematic exemplary cases in whichthe user travels the same streets but the way the geocoding measuring(e.g. HERE) gives the links causes a wrong behavior in the link method.Typically happens that big streets have two different linkIDs for thetwo direction of the street, or two streets are too near and HERE spotsthe user in the wrong one.

FIG. 24 shows a diagram illustrating schematic an exemplary occurrenceof the second case when the user goes once from point A to point B1(session S1), and once from A to B2 (session S2), as shown in FIG. 23 .If S1 and S2 have enough links in common (the user travels the same pathbut ends up in different places) the two trips are clustered together inthe Link method but not in the Start Stop method (in the cases in whichthe stop points are not enough near).

FIG. 25 shows a diagram illustrating schematic an exemplary embodimentvariant of the Bag of Links (BOL), which does not generate clusters, soa direct comparison on how the trips are agglomerated cannot beperformed. However, a good inspection on this method can be doneconsidering the get_familiarity process, respect to the otherget_familiarity of the other embodiment variants. The case in which theBOL embodiment variant becomes useful is when the user does a new tripusing only link that has already travelled in each of the previoussessions, but without covering the 80 percent of the shortest of thesesessions. In this case the start and stop points are far away so theget_familiarity start stop will return 0, also the number of links incommon are not enough to cover the 80 percent of links so also theget_familiarity of the link methods will return a low score. This newmethod instead will give a maximum scores of 1 (see FIG. 23 ).

FIG. 26 shows a diagram illustrating schematic an exemplary measuring ofa trip using an appropriate trip summary. When a TMR 113 request isreceived live, the system 1 respectively the TMR 113 checks if a useralready annotated or corrected a similar trip. Consequently, the system1 must be able to efficiently retrieve historical annotated trip dataand define a trajectory similarity measure. Since the TMR 113 liverequest contains a representation of the trip with 19 points, in thepresent embodiment variant, it makes sense to store this representationfor each annotated trip, partitioned by a user identifier. This can e.g.be done in a database or a filesystem (e.g. one row per trip). The userannotation preferably can e.g. be stored together with the trip summary.This trip summary can be built/updated in batch using, for example,Databricks (e.g. nightly). The embodiment variant can imply informationavailability within 24/48 h from user annotation. Existing facilitiesand other approaches can be considered as well.

FIG. 27 shows a diagram with a TMR baseline (given by the straight line)illustrating schematic an exemplary weighting of the parameters andevaluating the performance under TMR 113. The multiclass probabilitiescan e.g. be weighted less than the annotation probability. This is inline with the fact, that, if the user corrected a trip in the past and asimilar trip was observed by the system 1, the user should be trusted.The proposed value for the weight is 0.4.

FIG. 28 shows a diagram illustrating an exemplary embodiment variant ofthe DPD 112, which can be used for the trip familiarity detection 115,and which can e.g. comprise the following technical steps performed bythe system 1 and the trip familiarity detection and measuring 115,respectively: (1) Collect user history, (2) Cluster similar trips, (3)Define centroid trip, (4) New trips arrives: seek match with existingclusters, and (5) Check cluster DPD label. In FIG. 76 , (i) N is thetotal number of sessions with DPD score in the cluster, (ii) D_(i)∈[0,1]P₁∈[0,1] and X_(i)∈[0, 1] are final confidence scores returned by DPDfor each sessions (including enter/exit and BT connection), and (iii)cluster scores can be also generated from user annotations (Truth) oreventually from a combination of both sources.

FIG. 29 shows a diagram illustrating the exemplary objective of thefamiliarity score to create a measure for scoring purposes on how much auser travel on familiar roads. This can e.g. require the three differentmethods, as illustrated by FIG. 29 , i.e. (1) Clustering through linkID,(ii) bag of links: linkIDs frequency, and (iii) start & stop.

FIG. 30 shows a diagram illustrating an exemplary realization of thestart&stop method, as a powerful approach.

FIG. 31 shows an exemplary overview of a possible general architectureof the trip familiarity detection and measuring.

FIG. 32 shows a diagram illustrating an exemplary realization of anembodiment variant using a similarity prefilter technique, in particularfor TMR 113, where the data processing is preferably performed only on asubset of likely candidates. A trip is considered a valid candidate ofits start and end both lie within a certain radius from the start/end ofthe current trip (the one that is evaluating in a TMR live request). Theradius can e.g. be set to 500 meters for this example, based onempirical observation. Since user annotations can be in limited number(in normal operating conditions) and using the proposed similarityprefilter, the trajectory similarity is actually generated against asmall subset of trips, which is illustrated in FIG. 80 .

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 schematically illustrates an architecture for a possibleimplementation of an embodiment of the system and method for automatedtransportation mode recognition 113 based on sensory data 3/31/32/33measured by a plurality of sensors 102 of a cellular mobile device 10 ofa user 6. The plurality of sensors 102 at least comprise each anaccelerometer 1025 and a GPS sensor 1024. The mobile device 10 compriseone or more wireless connections 105. By at least one of the wirelessconnection 105 the cellular mobile device 10 acts as a wireless node221, . . . , 225 within a cellular data transmission network 2 by meansof antenna connections of the cellular mobile device 10 to the cellulardata transmission network 2. The plurality of sensors 102 are connectedto a monitoring mobile node application 101 of the mobile device 10. Themonitoring mobile node application 101 captures usage-based 3 and/oruser-based sensory data 3 of the cellular mobile device 10 and/or theuser 6 of the cellular mobile device 10.

The mobile device 10 measures time series 311,312,313;321,322,323 ofsensory parameter values based on measuring parameters 3 obtained fromthe mobile device's 10 sensors 102. The measuring parameters 3 comprisetime series 311,312,313 of sensory parameter values 31 of a 3-axisaccelerometer 1025/102 and time series 321,322,323 of sensory parametervalues 32 of GPS-based speed measurements of a GPS sensor 1024 /102. TheGPS sensor 1024/102 measures the mobile device's 10 longitude 10241,latitude 10242 and altitude 10243 position by measuring different speedof light delays in the signals coming from two or more satellites.

The measured time series 31 1,312,313;321,322,323 of sensory parametervalues trigger the automated transportation mode recognition 113 asinput feature values 1132 to a gradient boosting machine-learningclassifier 1131. The transportation modes 1133 at least comprise themodes (i) public transportation 11331 and/or (ii) motorcycle 11332and/or (iii) cycling 11333 and/or (iv) train 11334 and/or (v) tram 11335and/or (vi) plane 11336 and/or (vii) car 11337 and/or (viii) skiing11338 and/or (ix) boat 11339. The transportation mode recognition 113generates a transport mode label 1134 fora transport mode movementpattern 11351 of a trip 1135.

In a learning phase, which also can be dynamically or periodicallyapplied, as FIG. 2 shows, a supervised learning structure 1136 isapplied to the gradient boosting machine-learning classifier 1131 duringa supervised learning phase. The transport mode movement patterns 11351of measured trips 1135 are stored to a trips database 33, The mobiledevice 10 measures sensory movement parameters 311,312,313;321,322,323based on measuring parameters obtained from sensors 102 of mobiledevices 10 of heterogeneous set of users. The transport mode movementpatterns 11351 of a trip 1135 are identified from the measured sensorymovement parameter values 311,312,313;321,322,323. Each recognized tripcomprises measured sensory movement parameter values311,312,313;321,322,323 of GPS positions by the GPS-sensor 1024/102, andof acceleration forces being applied to the mobile device 10 on allthree physical axes by a 3-axis accelerometer 1025/1902, and ofoperating system activities parameter values of an operating system ofthe mobile device 10, and a transport mode label value 1134. Trips 1135with transport mode labels 1134 detected by the gradient boostingmachine-learning classifier 1131 are fed into a user back-loop 1136 fordynamic correction by a user associated with the respective trip 1135and saved to the trips database 33 by updating the learning transportmode movement patterns of measured trips 1135 in the trips database 33.As an embodiment variant, the trips database 33 is monitored by thesystem automatically detecting changes in the trips database 33, whereinupon detecting a change in the trips database 33, the supervisedlearning phase 1136 is reinitiated. In particular, the trips databasecan be updated continuously and/or dynamically based on the applied userback-loop. Further, for the supervised learning phase, the trips of thetrips database 33 can e.g. be preprocessed by filtering out transportmode movement patterns, which have a time duration shorter than 1 minuteand/or comprise less than 30 GPS positions and/or do not have a propertransport mode labelling. For the supervised learning phase, the tripsof the trips database 33 can e.g. preprocessed by filtering outtransport mode movement patterns having duplicated GPS locations bytimestamp and/or transport mode movement patterns with GPS locationshaving negative speed and/or transport mode movement patterns having GPSlocations with an accuracy>50 m.

It is important to note, that the present disclosure focus on smartphonedata used for transportation mode detection (TMR). Automated TMR is anessential technical requirement for many applications, includingtransportation monitoring, optimization, analysis and studies, urbanplanning, health monitoring, computer supported elder-care,epidemiology, dynamic usage-based risk-transfer, electronic trafficguidance systems, navigation systems and mobile applications etc. Withthe knowledge of travelers' transportation mode, targeted and customizedadvertisements may be sent to their devices, allowing a new way aufautomation. The output signaling of automated TMR systems is also usefulfor the development of context aware cell phones that sense the currentcontext and adapt their behavior accordingly. Also, if the precisetransportation modes of individual users are discovered, it is possibleto provide a more realistic picture of travel demand. The TMR detectionmay also help to monitor and detect the environmental impact of measuredtravel patterns, such as carbon footprints of users, and track the dailystep count of users and amount of calories they burn. Anotherapplication is the detection of real-time traffic state becausecompanies such as Google or TomTom collect data from mobile phones inorder to estimate the traffic speed on roads. In summary, automated TMRsystems have a plurality of technical applications and solve a varietyof technical problems.

The fact that transport mode detection based on GPS tracking data istechnically extremely challenging, especially when the GPS tracking datais unlabeled, i.e., there is no information regarding the transport modeused during a trip, it can technically be advantageously to use thepresent automated TMR system 113 as a part of a more integratedtechnical environment, such as in combination with automated tripfamiliarity detection 114 (as discussed below), and/or automatedpassenger-driver detection 112 by sensory parameter value patterns ofmobile devices 10, as also exemplary shown in FIG. 1 .

In further embodiment variants, the signaling output of the presenttransport mode recognition system 113 can e.g. be used as steering orinput signals for Advanced Driver Assistance Systems (ADAS), trafficcontrol systems, navigation system, usage-based risk measuring andmonitoring systems etc. For example, the system 1 can e.g. comprise oneor more automated first-tier risk-transfer systems 12 (automated primaryinsurance systems) and one or more automated second-tier risk-transfersystems 13 (automated reinsurance systems). The automated first-tierrisk-transfer systems 12 can comprise at least one electronic first-tierresource-pooling system 121 and the automated second-tier risk-transfersystems 13 can e.g. comprise at least one electronic second-tierresource-pooling system 131. Resource-pooling systems 121/131 aresystems for automated, electronically steered pooling of resources fromassigned risk exposed occupants/drivers/passengers 6/61/62, therebytransferring a defined risk associated with the risk exposed user 6 tothe automated first-tier and/or second-tier systems 12/13, wherein theoperation of the transferred risk is defined by risk-transfer parameters122/132, e.g. predefined by means of predefined parameters given byrisk-transfer policies, and wherein in case of triggering the occurrenceof the defined risk at a user 6, an occurring and detected loss of theconcerned risk exposed user 6 is distinctively covered by the automatedresource-pooling systems 121/131 by triggering the specific transfer ofresources from the resource-pooling system 121/131 to the concerned riskexposed user 6, e.g. through appropriate signaling based on generatedpayment transfer parameters 123/133. The operation of such a system 1will be described in detail below. The risk-transfer parameters 122/132can e.g. comprise parameters defining physical measuring parameters todetect the occurrence of a risk event at the risk exposed user 6, bymeans of the system 1 and/or time—or amount related threshold values.The risk exposed user 6 can be any type of person and the risk can e.g.be associated with vehicle-or car-driving or traffic risk, e.g.associated with a driver or passenger. A risk is related to theprobability for the occurrence of an impacting event in relation torisk-exposed user 6. The automated system 1 can e.g. include at least aprocessor and associated memory modules. The operation of the system 1is controlled, monitored and steered by the electronic control device11, in particular generating appropriate signaling and steering theactivation and interworking of the various components of the automatedsystem 1. The automated system 1 can also include one or more displayunits and operating elements, such as a keyboard, and/or graphicpointing devices, such as a computer mouse. The system 1 is a technicaldevice inter alia comprising electronic means used in the field ofcomputer and data processing technology, telematic technology andautomated risk transfer or insurance technology. The invention seeks totechnically capture, manage and automate complex related operations ofmonitoring devices.

Transport Mode Recognition (TMR) 113

The architecture of the Transport Mode Recognition (TMR) systemrepresents a Machine Learning (ML) based solution: a collection oflabeled trips performed by a heterogeneous set of users is measured andanalyzed to extract a set of features that is used to train a supervisedmulticlassification Machine Learning structure. The output of the pureMachine Learning structure is then postprocessed by a set of hard codedrules, in order to avoid the algorithm to make clearly incorrect orinsufficiently confident predictions. An additional add-on module basedon Trip Familiarity can recognize the user's routines and can beactivated to increase the model performances. The reliability of theautomated transport-mode recognition increases as more data points areaccumulated, where the accumulation can be performed by an automatedprocess, das described below. As an embodiment variant, if the MLstructure fails to recognize the mode of transport correctly, users canhave the option to manually correct the predicted transport mode in thesystem. The changes can be automatically detected, and the supervisedmodel is retrained in order to avoid repetition of the same mistake andimprove the overall performance: the model's prediction capabilitiesimprove in a continuous cycle (see FIG. 2 ).

The measuring data used to build the TMR system consist of trips beingcaptured and hold by appropriate data structures, wherein each tripcomprises the following measuring parameters measured and recorded by amobile device: (i) GPS Positions, (ii) 3-axis Accelerometer, (iii)Operating System Activities, and (iv) Transport Mode Label annotation.The trips can comprise, for example, measuring data for transport modesas car, motorcycle, tram, bus, cycling, skiing, train, plane, boat, orothers.

The system can e.g. apply a data filtering and preprocessing. Forexample, of the trips, data can be filtered out based on the followingconditions: (i) At least one minute long, (ii) At least 30 GPSpositions, and (iii) Exactly transport mode annotation. As a furtherpreprocessing step, trips comprising duplicated GPS locations bytimestamp, GPS locations that have negative speed and GPS locations thathave accuracy>50 m can e.g. be removed.

The system can further comprise a data enrichment process. For example,the trip data can be enriched. As an embodiment variant, the enrichmentprocesses can also be based on external APIs of third party providers.The performed enrichments can e.g. comprise: (i) Route matching, inorder to understand if the trip was performed on a road, (ii) Queryusing a Geographic Information System (GIS) geometries near the trip,the GIS capturing spatial and geographic data, (iii) Public transportmatching. For the data enrichment, as a variant, the enrichment does notneed to be performed on the full GPS track to perform, but (for exampledue to some APIs limitations) only a subset equally spaced GPS positionscan be used.

The measured and generated features for the available trip data can e.g.comprise:

(1) GPS features comprising over the array of measured GPS speeds (i)Average, (ii) Standard deviation, and (iii) Percentiles from 0 to 100,with step 10 (so percentile 0, 10, 20 . . . ). Over the array ofmeasured GPS altitudes of a trip the standard deviation, and over themeasured array of GPS accelerations (i) the standard deviation, and (ii)the variance of the array obtained by measuring the angle betweentriplets of consecutive GPS points. GPS acceleration is defined hereinas the ratio between the following two arrays: (i) Speed differencebetween a GPS sample and the preceding sample, (ii) Time difference (inms) between a GPS sample and the preceding sample.

(2) Accelerometer features: If two or more accelerometer samples havethe same timestamps, the last one can e.g. be selected with respect toarray order. The acceleration norm array is then computed and, theaverage of this array is removed from all the same array. From the normarray, the following parameters can be extracted: (i) The percentilesfrom 0 to 100, with step 10; (ii) The interquartile range, which is thedifference between the 75th and the 25th percentile.

(3) Features based on operating system activities: Two features usingthe operating system activities can e.g. be selected, based on thefollowing criteria: (i) “Forward integral” processing is chosen becauseof the event-wise behavior of the activity labels, and since it'sgenerally the most informative feature, (ii) “Count as most probable”processing can e.g. be chosen for simplicity. An activity event can bedefined as a measuring object with a unique timestamp and a map oflabels with probabilities (if a label is absent is considered to havezero probability). The two features are calculated for each possiblelabel. Labels can be normalized to the Android names: ‘Automotive’,‘Cycling’, ‘OnFoot’, ‘Running’, ‘Stationary’, ‘Unknown’, ‘Walking’,‘Tilting’ for feature vector naming compliance between the two operatingsystems. The “forward Integral processing” can be implemented byassuming that a label probability is valid until the next event. Eachlabel probability can be multiplied by the milliseconds elapsed untilthe next event, or until the end of the trip for the last receivedactivity event. This must be done for each label of the possible labellist. The results of these multiplications can be summed up for eachlabel, and each sum can be divided by the difference between trip endtime and the first activity event time, both in milliseconds. If a labelis never returned, the corresponding feature should be set to zero. So,if there are no activities at all for a trip, all the features should beset to zero. The “count as most probable” processing can e.g. beimplemented, in that for each possible label the number of events iscounted, in which the label was the most probable one, where the countis then divided by the total number of events (or the number of uniquetimestamps). In case of a 50/50 draw, the most probable can be selectedin any way.

(3) Public transport features: Given the set of GPS locations, candidatestops can e.g. be identified as sequences of points that fulfill thefollowing conditions: (i) Speed<=3 m/s, (ii) Sequences are longer than 5seconds. The calculation can be performed after applying a movingaverage with window length 9 over the array of speeds. This means thatevery sample is replaced by the average of the sample itself and the 4samples before and after. For each of these candidate sequences, theaverage latitude and average longitude can be generated, obtaining acandidate stop position for each sequence/stop (see FIG. 3 ). Using thepublic transport algorithm inputs and outputs, some additional featurescan be generated, comprising: (1) the number of candidate stops of thetrip (trajectory stops), (2) the number of candidate stops of the trip(trajectory stops) divided by the cumulated sum of haversine distancesbetween the 16 sampled GPS points, ordered increasingly by time, inmeters, (3) the number of suggested stops for the best matching APIsuggestion, (4) the cumulated haversine distance of the suggestionstops, in order of traversal, divided by the cumulated haversinedistance of the sampled GPS points, (5) the cumulated haversine distanceof the candidate stops, divided by the cumulated haversine distance ofthe 16 sampled GPS points (see point 2), and (6) the percentiles from 0to 100, with e.g. step 10, of the minimum distances from the suggestionstops to the candidate stops (this is the standard public stopalgorithm). These features can be generated for all the suggestions, butthe ones selected are the ones regarding the suggestion with minimumdistance between suggestion stops and candidate stops.

(4) Route Matching (RM features: Route matching features can e.g. begenerated in order to estimate, based on the sampled GPS points, if thetrip was not performed on a road. Two statistical descriptors can e.g.be generated over the trace points confidences: (i) the average of theconfidences, and (ii) the variance of confidences.

(5) Geofencing (GFE) features: Using the geometries returned by generalgeofencing, the features used by the proximity search can e.g. begeneralized. Basically, given the enrichment for the sampled GPS points,the percentage of points can be generated having geometries within 10,20, 30 . . . 100 meters. This possibly includes points within ageometry, having distance<0. These distributions are can e.g. begenerated for: (i) percentage of points seeing only roads within a XXradius (XX from 10 to 100, with step 10), (ii) the percentage of pointsseeing only rail tracks within a XX radius, (iii) the percentage ofpoints seeing or roads or rail tracks within a XX radius, (iv) thepercentage of points within an airport (negative geometry distance), and(v) the percentage of points within an airport (negative geometrydistance).

The set of features described above can e.g. be used to feed themachine-learning gradient boosting structure (e.g. LightGBM) with thefollowing hyperparameters configuration: n-estimators=225,learning-rate=0.03, max-depth=30, num-leaves =50, subsample=0.8,subsample=0.7, and min-sum-hessian-in-leaf=5.

For the hardcoded filtering rules, for example, some custom logic can beadded after the Machine Learning classifier in order to limit unwantedmistakes. The first set of rules works on the trip features, to correctpredictions that are clearly wrong. The second set of rules acts only onthe trip modes with low precision, adjusting the predictions that have alow level of confidence, with the goal to reduce the false positiverate.

The rules based on trip features can e.g. comprise: (i) If GFEWater0>0.5mark this trip as ‘boat’, (ii) If GFEWater0<0.2 andTMR-prediction=‘boat’, mark this trip as ‘other’, (iii) IfSpeedQuantile90>150 m/s, mark the trip as ‘plane’, and (iv) IfTMR-prediction=‘plane’ and SpeedQuantile100<20 m/s and GFEAirport=0,mark this trip as ‘other’. A rules based on model confidence can e.g. beimplemented, where if Predicted Transport Mode>Score Threshold, theprediction will be changed to Fallback Transport Mode

Predicted Score Fallback Transport Mode Threshold Transport Mode Public0.80 Car Motorcycle 0.90 Car Cycling 0.88 Other Train 0.85 Other Plane0.85 Other

FIG. 4 shows exemplary performances achieved by the TMR system asillustrated by the confusion matrix and table of FIG. 4 and obtainedthrough a 5-fold Cross-Validation with a leave k-users out splittingtechnique, in order to reduce overfitting.

As an embodiment variant, in addition to the supervised learningstructure for TMR, a trip similarity strategy can be applied to furtherraise the TMR performances and accuracy. The idea is leveraging userannotations over previous similar trips, if any, and use thisinformation to correct TMR labels, if needed. Thus, to find similartrips, a clustering algorithm can e.g. be run on the following features:(i) Distance between Trips Starting point*, (ii) Distance between TripsEnding point*, (iii) Trip Distances (* distance between start/endingpoints of the two trips is repeated after reversing one trip, to ignorethe travel direction).

In order to improve the recognition between Car and Moto modes oftransport, in an even further embodiment variant, a dedicated binaryclassifier can be applied. Leveraging the used Deep Learningarchitectures, the dedicated binary classifier aims to extractdiscriminating features directly from smartphone sensor time series: (i)3-axis accelerometer, and (ii) GPS Speed.

As further data preprocessing, the time series can e.g. pass through thefollowing preprocessing steps, before being ingested by the neuralnetwork: (i) Rotation of the 3-axis accelerometer from the smartphonereference system to the vehicle reference system, (ii) Alignment betweenaccelerometer and GPS, sharing a common 10 Hz sampling grid, (iii) Eachtrip is split into multiple 5 minutes long mini-trips. The final inputto model can e.g. be then a 4-dimensional time series, with a fixedlength of 3000 timesteps (5 minutes*10 Hz).

An exemplary model architecture is shown in FIG. 3 . Further,performances can e.g. be assessed through a 5-fold Cross-Validation witha leave k-users out splitting technique, leading to the results as shownin FIG. 4 .

It has to be noted, that in various embodiment variants, thearchitecture of the Transport Mode Recognition system is very flexibleand can be performed on a variety of environments, as e.g. theDatabricks environment. The example of the Databricks environment hase.g. the advantages: (1) Having a shared codebase for quick prototypingand testing; (2) Enabling the reuse of the output code directly in thesolution deploy; and (3) Get advantage of native Spark parallelism inorder to perform multiple experiments and test different approaches.Other environments may have different advantages. Databricks is atechnical environment growing out of the AMPLab project at University ofCalifornia, Berkeley that was involved in making Apache Spark, anopen-source distributed computing framework built atop Scala. Databricksprovides inter alia a web-based platform for working with Spark, thatprovides automated cluster management and IPython-style (InteractivePython) notebooks, i.e. providing a command shell for interactivecomputing in multiple programming languages, in particular the Pythonprogramming language, offering introspection, rich media, shell syntax,tab completion, and history.

For the present invention, the analytics pipeline architecture wasshaped to reflect, the flow of the live solution. The used pipeline canbe separated in the following logical components: (1) Extract data fromTMR campaign and IoT Platform (positions, sensors, annotations debugdata), and enrich it with GIS services (HERE); (2) Extractdescriptors/features from valid trip data; and (3) Learn a TMRclassification model in a supervised learning setup. These three stepsare the object of the following description. The final output of thelast step is a classification model structure that can be consumed by aproduction service. This structure is a standard scikit-learn objectthat can be deployed e.g. in any Python enabled architecture. FIG. 7shows a block diagram illustrating schematic an exemplary overview ofthe architecture of the Transport Mode Recognition system part of system1, comprising (i) data extraction, data extraction and filtering, (iii)Position sampling, (iv) Candidate stops, (v) Data enrichment, (vi)Feature description, in particular GPS features, Accelerometer features,Feature based on operating system activities, Public transport features,RME features, and GFE features, (vii) Machine learning, (viii) Hardcodedfiltering rules, and (ix) Early detection.

For the data extraction, trip data have to be merged from differenttables. As a trip identifier, start/stop boundaries can e.g. be used,for example such ones uploaded as debug data by an appropriate debugmodule or application. This data is e.g. be uploaded to a container andbe copied to corresponding tables stored in a data repository. Inprinciple, one could use any trip boundary information. For example,JSON (JavaScript Object Notation) trip boundaries can be used in orderto ensure consistency with the debug application logic, but this is nota constraint. Moreover, additional data can be extracted from the JSONwhich may not contained in the tables in a first time period, mostnotably the OS Activity and TMR library labels. Since an embodimentvariant may use only the OS Activity labels among the two, and theselabels can be uploaded in the normal application data flow, the debugJSON data is not a requirement for the inventive solution (cf. FIG. 8 ).

For the data extraction and filtering, the trip boundaries can e.g. bemerged with the following exemplary data sources: (1)analyticsmodel_np0.positions for the GPS locations (Note that “np0”refers to the environment and can change depending on user or testingframework), (2) analyticsmodel_np0.userannotations for the ground truthprovided by Coloride users, (3) OS Activities contained in the JSON, (4)(optional) analyticsmodel_np0.accelerometers, and (5) (optional)analyticsmodel_np0.deviceevents. Of these trips, data are filtered outby the system 1 based on the following conditions: (1) At least oneminute long, (2) At least 30 GPS positions, and (3) Exactly one userannotation. As a preprocessing step, the system 1 can remove duplicatedGPS locations by timestamp, GPS locations that have negative speed andGPS locations that have accuracy>50 m. Motivation for the latter choiceis illustrated by FIG. 9 , which shows the performance variation of theTMR classifier when varying the minimum accuracy threshold. Since theperformance variation is not strong, compliance can be kept with thepreviously used threshold.

For the position sampling, in order to perform the TMR live call thesystem 1 needs to perform data processing on a subset of data points,since an arbitrary amount of data cannot be sent in a remote synchronouscall. However, it should be noted that the process can e.g. also beimplemented to use the full trip data. Based on the analysis of theperformance over the number of sampled GPS locations, the points to besampled can e.g. be set to 16 points equally spaced over the GPSlocations array. Other sampling strategies could also be used, however,in the present example did not show a significant performance gain.Also, though more points could be sampled, there is up-to-now noevidence suggesting that sampling more than 16 points may be beneficial.16 points can e.g. be chosen because it is the HERE Geofencing API limitfor a batch call (cf. FIGS. 10 a and 10 b ).

Candidate stop extraction can e.g. be performed by the system 1identically to the current TMR implementation. Given the set of GPSlocations, candidate stops are identified as sequences of points thatfulfill these conditions: (i) Speed<=3 m/s, and (ii) Sequences arelonger than 5 seconds. The data analysis is performed after applying amoving average with window length 9 over the array of speeds. This meansthat every sample is replaced by the system 1 by the average of thesample itself and the 4 samples before and after. For each of thesecandidate sequences, the system 1 generates the average latitude andaverage longitude, obtaining a candidate stop position for eachsequence/stop (cf. FIG. 11 ).

For the data enrichment, trip data is then enriched by the system 1 withe.g. external APIs (Application Programming Interface), for exampleusing HERE services and an appropriate proximity search. However, alsoother appropriate map providers can be used of the map can be generatedbased on own sensory and/or other data. A depiction of the enrichmentcan be seen in FIG. 12 . Apart for baseline data, the performedenrichments can e.g. comprise: (1) Route matching, in order tounderstand if the trip was performed on a road. In the experimentalsetup the service used is HERE Route Match Extension (RME). Alternativeservices or a normalized data source can be used, (2) Query ofGeographic Information System (GIS) geometries near the trip, performedusing a HERE GFE API/layers. This step is basically a generalization ofthe GFE approach used in a possible TMR solution (same source, moregeneral features), and (3) Public transport suggestions, in this caseHERE Routing API. In the analytics data processing pipeline of thesystem 1, data can e.g. be written on filesystem after this stage. Thiscan happen for the overall data, which can be slow, or incrementally ona monthly basis. This is performed since the enrichment step is the“slow” one.

For the features description, the system 1 generates a number offeatures based on all available or historic trip data (e.g. see featureextraction illustrated in FIG. 13 ). The computed feature list is asuperset of the used features. Below, the features actually used aredescribed in the TMR solution, so generated features that are notdescribed are typically mostly out of scope. The implementation of suchextractions is mostly contained in the second step of the TMR analyticspipeline. In the production solution, if the constraint is a live TMRcall, some of the features must be generated locally on the used phone10 and sent together with the TMR API call. Alternatively, if TMR can beperformed asynchronously, these features can also be generated as soonas trip data lands on the IoT platform of the system 1.

Regarding the GPS features, over the array of GPS speeds, the followingfeatures can e.g. be generated: (1) Average, (2) Standard deviation, and(3) Percentiles from 0 to 100, with step 10 (so percentile 0, 10, 20 . .. ). In the exemplary Databricks implementation, the percentile NumPyfunction can e.g. be used, with the interpolation parameter set to“nearest”, whereas the known NumPy function provides a large number ofpredefined mathematical operations including standard trigonometricfunctions, functions for arithmetic operations, handling complexnumbers, etc.

Over the array of GPS altitudes, the following feature can e.g. begenerated: Standard deviation. Further, GPS acceleration can beimplemented as the ratio between the following two arrays: (1) Speeddifference between a GPS sample and the preceding sample, and (2) Timedifference (in ms) between a GPS sample and the preceding sample.Finally, over the resulting array of GPS accelerations, the followingfeature can be generated: Standard deviation. A measure of directionvariance of the trip can also be generated, following of the pipelineimplementation. Zero values from the bearing array can e.g. be removed.

Regarding the accelerometer features: If two or more accelerometersamples have the same timestamps, select the last one w.r.t. to arrayorder. The acceleration norm array can then be generated and, theaverage of this array can be removed from all the same array. From thenorm array, some statistics can be extracted comprising: (i) Thepercentiles from 0 to 100, with step 10, (ii) The interquartile range,which is the difference between the 75th and the 25th percentile.

Regarding the feature based on operating system activities: Two featuresusing the operating system activities can be selected, with thefollowing rationales: (1) ForwardIntegral can be chosen because of theevent-wise behavior of the activity labels, and since it's generally themost informative feature, and (ii) CountAsMostProb can be chosen forsimplicity. An activity event, as used herein, is an object with aunique timestamp and a map of labels with probabilities (if a label isabsent is considered to have zero probability). The two features aregenerated for each possible label. Labels can e.g. be normalized to theAndroid names: ‘Automotive’, ‘Cycling’, ‘OnFoot’, ‘Running’,‘Stationary’, ‘Unknown’, ‘Walking’, ‘Tilting’ for feature vector namingcompliance between the two operating systems. To perform a forwardintegral calculation, it can be assumed that a label probability isvalid until the next event. Each label probability can be multiplied bythe milliseconds elapsed until the next event, or until the end of thetrip for the last received activity event. This must be done for eachlabel of the possible label list. The system 1 sums the results of thesemultiplications for each label, and divide each sum by the differencebetween trip end time and the first activity event time, both inmilliseconds. If a label is never returned, the corresponding featureshould be set to zero. So, if there are no activities at all for a trip,all the features should be set to zero. Further, the system 1 performs acount as most probable calculation, where for each possible label thenumber of events is counted in which the label was the most probableone, and divide by the total number of events (or the number of uniquetimestamps). In case of a 50/50 draw, the most probable can be selectedin any way.

Regarding the public transport features, public transport algorithminputs and outputs are used to generate some additional features: (1)CandidateStopsCount: the number of candidate stops of the trip(trajectory stops), (2) CandidateStopsCountNormalized: the number ofcandidate stops of the trip (trajectory stops) divided by the cumulatedsum of haversine distances between the 16 sampled GPS points, orderedincreasingly by time, in meters, (3) PublicRoutingNumStops: the numberof suggested stops for the best matching API suggestion, (4)PublicRoutingDistRatio: the cumulated haversine distance of thesuggestion stops, in order of traversal, divided by the cumulatedhaversine distance of the 16 sampled GPS points (see point 2), (5)PublicRoutingCandidateDistRatio: the cumulated haversine distance of thecandidate stops, divided by the cumulated haversine distance of the 16sampled GPS points (see point 2), and (6) The percentiles from 0 to 100,with step 10, of the minimum distances from the suggestion stops to thecandidate stops (this is the standard public stop algorithm). Thesefeatures are calculated for all the suggestions, but the ones selectedare the ones regarding the suggestion with minimum distance betweensuggestion stops and candidate stops.

Regarding the RME features: RME features are generated in order toestimate, based on 16 GPS points, if the trip was not performed on aroad. Two statistical descriptors are generated over the trace pointsconfidences: (1) The average of the confidences using e.g. animplemented RMESampledTracePointsConfMean routine, and (2) the varianceof confidences, using e.g. an implementedRMESampledTracePointsConfVariance routine.

Regarding the GFE features: Using the geometries returned by the GFE API(e.g. the HERE GFE API), the system 1 can generalize the features usedby the Proximity Search. Basically, given the enrichment for the 16points, the system 1 generates the percentage of points havinggeometries within 10, 20, 30 . . . 100 meters. This possibly includespoints within a geometry, having distance <0. These distributions arecomputed for: (1) GFERoadOnlyXX generating percentage of points seeingonly roads within a XX radius (XX from 10 to 100, with step 10), (2)GFERailOnlyXX generating percentage of points seeing only rail trackswithin a XX radius (see point 1), (3) GFERailRoadXX generatingpercentage of points seeing or roads or rail tracks within a XX radius(see point 1), and(4) GFEAirport0 generating percentage of points withinan airport (negative geometry distance). The exhaustive way of mappinggeometries to originating points is to do a separate call for each ofthe 16 sampled GPS points. However, this can be expensive in terms ofresources. A batch call with all the 16 points together can e.g. beperformed, and then the geometries mapped back to the originating pointsby minimizing the haversine distance between the points and thenearestLat/nearestLon attributes for each geometry (for differences anddetails, see the batch version variant in the first step of the TMRpipeline—where GFE_API_Call should be replaced above in step1, andGFEFeats should be replaced in step2). It is to be noted that the secondapproach is less expensive but it's also less exact, so the overallperformance can be slightly lower.

After the feature generation phase, the trip representation isserialized to the filesystem. For selecting the above described featuresfrom the larger generated features pool, a cross-validated RecursiveFeature Elimination (see FIG. 14 ) can be used in order to get anestimate of the optimal feature set, averaging results over multipleexperiments in a leave-k-users-out setup. Feature importance can beassessed for each classification setup (see below).

For the machine learning, in order to maximize classifier performancesand fulfill the technical requirements, a two-stage classifier can e.g.be built. The first classification stage is a specialized “car”/“nocar”detection. This step maximizes performances over the transportation modeof main interest. Trips that are classified as “car” in the first stepare permanently marked as “car”. Trips that are not classified as carare then fed to a multiclass classifier that tries to assign the correcttransport mode over the available classes. If the multiclass predicts“car” when the first step did not, we mark the trip as “unknown”. Thisis motivated by precision measure evaluation. The classifier can e.g. betrained, leveraging TMR NP0 pilot data, over the following transportmodes: car, train, public transport, bicycle, motorcycle, skiing, plane.The exemplarily chosen classification algorithm is Random Forest. Otheralgorithms are also imaginable. Motivation for this choice can e.g. stemfrom the need of controlling overfitting in the model, havingprobability estimates in the prediction. Moreover, this algorithm hasthe advantage of providing a good method for estimating featureimportance. For tuning the algorithm parameters a grid exploration wasperformed after the feature selection phase (see FIG. 15 ). An exemplaryconfiguration is: (1) 250 trees with maximum depth 8 for the binaryclassifier, and (2) 250 trees with maximum depth 10 for the multiclassclassifier. After a successful training, models and results can e.g. beserialized for consumption, e.g. by a live service.

In addition to the full track classification approach described above,an early classification can e.g. be performed. Early check recognitioncan e.g. be performed among all trip modes, using, for example, thefirst 5 and 10 minutes of the trip. The following table shows F1-Scoreperformance degradation using the first n minutes of the trip, respectto the Full Trip performances:

5 minutes 10 minutes Boat −4.0% −4.0% Car −1.0% −0.5% Cycling −1.2%−1.2% Motorcycle −8.5% −4.2% Plane −1.3%  1.3% Public −3.5% −7.0% Skiing 2.2%  2.2% Train −6.5% −5.4%

The final and best performance of the TMR algorithm can e.g. be obtainedby the single multiclass gradient boosting classifier. Performance canbe evaluated in a leave-k-users-out cross-validated setup, in order toget a realistic performance projection, and it is inclusive of all thepost processing hardcoded rules. (see below)

Transport Mode Support Recall Precision Boat 12 100.00% 100.00% Car12710  98.68%  94.98% Cycling 407  71.74%  91.54% Motorcycling 851 53.94%  88.78% Other 13  30.77%   2.60% Plane 115  77.39%  88.12%Public 1000  77.90%  88.12% Skiig 349  93.70%  92.90% Train 316  82.59% 95.96%

Further, it is possible a make similarity add-on at the inventive system1. Thus, in addition to the supervised learning approach for TMR, a tripsimilarity strategy can be applied, for example, in order to furtherraise TMR performances. The additional approach can, for example,leveraging user annotations over previous similar trips, if any, and usethis information to correct TMR labels, if needed. This feature can beeasily integrated in a production API, where the requirement is to havethe 16-points representation of annotated trips available to the API,partitioned by user. The service can e.g. receive a new 16-pointsrepresentation of a trip, together with TMR probability output, andmatches this trip with similar annotated trips, if they exist.Similarity is calculated using a Euclidean pseudo-distance betweentrajectories. If one or more matches are found, a simple weightingalgorithm modifies the TMR probabilities based on the annotationevidence. The new most probable class is then chosen as the TMR label.FIG. 19 show an exemplary F1 score varying TMR label weight (probabilitymass assigned to the automatic label).

Trip Familiarity Score or Index Measuring

According to the present invention, there are different embodimentvariants to technically assign to users and to sessions a score of howmuch of them follow habits (i.e. familiarity score measuring 114). Thefirst two embodiment variants use a clustering method and then evaluatethe familiarity from the dimensions of the clusters (and the familiarityof sessions from the dimension of the clusters in which them areassigned).

Below, the used variants of clustering method and the scoring method aredescribed:

In a fist embodiment variant, which uses a set of links of each session(herein denoted as link version), the clusters are created usingjacquard similarity between the link of the sessions. Jaccard Similarity(coefficient) measures similarities between sets. It is defined as themeasured size of the intersection divided by the size of the union oftwo sets. In particular, the similarity between two sessions arecalculated in this way:

${{Sim}\left( {S_{1},S_{2}} \right)} = \frac{❘{L_{S1}\bigcap L_{S2}}❘}{❘{L_{S1}\bigcup L_{S2}}❘}$

where L_(Sx) is the set of links of the session x. The agglomeration isdone starting from one cluster for each session, and by agglomeratingclusters that have a similarity of at least 0.8. The similarity betweenclusters with more than one session in it is done by considering themaximum similarity between all the possible combinations of sessions.

In a second embodiment variant, using start and stop points of eachsession (herein denoted as Start and Stop version), the start and stoppoints of each the sessions are used for clustering. The distancesbetween two sessions are generated in the following way:

D(S ₁ ,S ₂)=hav(P _(A1) ,P _(A2))+hav(P _(B1) ,P _(B2))

where P_(Xn) is the start(A) or end(B) point of the session n, and hav() is the Haversine distance between two points. The Haversine distancemeasures the great-circle distance between two points on a sphere giventheir longitudes and latitudes. The agglomeration can e.g. be donestarting with a cluster for each session, considering as centroid of thecluster the couple start and end points of the session. The next step isdone by agglomerating the clusters with a distance of 300 meters orless, iteratively. Every time two clusters are joined the centroid ofthe cluster are recalculated with a simple average of latitude andlongitude of both A and B points of the centroids. Then anotheragglomeration is done like the previous but considering the centroidsdistance with the points matched in reverse way (start-points matchedwith end-points).

For the scoring generation of user familiarity and after the clustering,the Gini coefficient can be used on the dimensions of the clusters toassign to each user a familiarity score. The Gini coefficient measuresthe inequality among values of a frequency distribution (here thefamiliarity of trips). A Gini coefficient of zero expresses perfectequality, where all values are the same (for example, where all measuredpoints of the trip match). A Gini coefficient of one (or 100%) expressesmaximal inequality among values (e.g., for a large number of trips whereonly one trip has different measure points and all other trips havecomplete match, the Gini coefficient will be nearly one). Note that forlarger sets of trips, values close to one are unlikely.

The following relation gives a possible index, which can be used for thegeneration of the familiarity and familiarity score, respectively:

${{Fam}_{2}(U)} = {\sum\limits_{i}{{❘C_{i}❘}(\lambda)^{i}}}$

where |C_(i)| is the percentage of user session in the i-th cluster,taking the clusters in dimension order, decreasing. λ is a parameterbetween 0 and 1 that indicates how clusters are considered in theproposed scoring. This value defines the weight given to each cluster inthe final score, depending on the position of the cluster in theordering. For example, if the value is set to 0.5, the first clusterwill count 1, the second 0.5, the third 0.25 and so on. If the value isset to 1, each clusters is considered in the same way, if the value isset to 0, just the first cluster is considered. In an embodimentvariant, this value is stetted to 0.5. The main idea of this index is todesign a value that orders the users with the following order, given theclusters dimensions (x-axis: cluster number, y-axis: cluster dimension),as illustrated in FIG. 20 .

For comparison between the Gini index and the index used in thisembodiment variant, the used index is generated to adjust the fact thatthe first and the last two cases of the ordering wanted score 0 in theGini index, that is an acceptable value just for the last one case. InFIG. 21 the correlation between the Gini index and the used index isshown. As it can be seen, there is a set of value that scored 0 in Ginibut they assume a significative value in this new index. Further, it canbe seen that the correlation between this two indexes seems to show somekind of regularity in the couple of values. The graph shows that thereis some groups of points placed on the same line. This means thatfurther exploration can lead to some kind of clustering algorithm, thatuses a combination of this two indexes.

In any case, no general correlation can be overserved between the twoindexes because they have two different concepts behind. Gini definessome kind of variance of the cluster dimensions, the new index defines ameasure on how the sessions is distribute into the clusters, focusing onthe main clusters. Both can be considered as measures of the userFamiliarity. Finally, to score for the session familiarity, thefamiliarity score for a session is measured as the relative dimension ofthe cluster in which the session is placed, generated as the divisionbetween the session in cluster and the total sessions of the user.

A third embodiment variant of Familiarity (denoted herein as “Bag ofLinks” embodiment variant (BOL)) starts from a scores of familiarity foreach link to calculate familiarity of sessions and users. A score offamiliarity for each link of each user is generated as the percentage ofsessions of the user in which the link appears. The session familiarityis generated as the average of the links scores in the session, the userfamiliarity is generated as the average of the scores of the linkstravelled by the user.

To compare the three proposed embodiment variants, the following can beobserved: In the first two embodiment variants the familiarity dependson the way the sessions are clustered. After an inspection on theresults, the cases in which the two methods give different results arethe following. The user goes from the same point A to the same point B,but passing through different links (see FIG. 22 ). This behavior causeslow aggregation in Link familiarity variant and high aggregation inStart Stop variant. In the dataset it has been spotted some cases inwhich the user travels the same streets but the way the geocodingmeasuring (e.g. HERE) gives the links causes a wrong behavior in theLink embodiment variant. Typically, it can happen that big streets havetwo different linkIDs for the two direction of the street, or twostreets are too near and the geocoding measuring (e.g. HERE) spots theuser in the wrong one. (see FIG. 23 )

A second case happens when the user goes once from point A to point B1(session S1), and once from A to B2 (session S2), as shown in FIG. 23 .If S1 and S2 have enough links in common (the user travels the same pathbut ends up in different places) the two trips are clustered together inthe Link method but not in the Start Stop method (in the cases in whichthe stop points are not enough near). (see FIG. 24 )

The Bag of Links (BOL) embodiment variant does not generate clusters soa direct comparison on how the trips are agglomerated cannot beperformed. However, a good inspection on this method can be doneconsidering the get_familiarity process, respect to the otherget_familiarity of the other embodiment variants. The case in which theBOL embodiment variant becomes useful is when the user does a new tripusing only link that has already travelled in each of the previoussessions, but without covering the 80 percent of the shortest of thesesessions. In this case the start and stop points are far away so theget_familiarity start stop will return 0, also the number of links incommon are not enough to cover the 80 percent of links so also theget_familiarity of the link methods will return a low score. This newmethod instead will give a maximum scores of 1 (see FIG. 25 ).

To realize the different embodiment variants, different libraries cane.g. be used to generate the familiarity and relative examples of usage.Each libraries can require a specific input and retrieve the same outputcomposed of three different dataframes. Exemplary dataframes my comprisethe following composition: (i) familiarity_user: UserID: User_ID,SessionSize[ ]: Array containing the dimensions of clusters of thatuser, Familiarity: Index calculated with Gini index, Familiarity_v_2:Index calculated with the new index (described above); (ii)familiarity_session: UserID: User_ID, SessionID: Session_ID,familiarity_sess: Session familiarity, it is the relative dimension ofthe cluster in which the session is placed (session in cluster/totalsessions of the user), and (iii) clusters: UserID: User_ID, Cluster:Generated identifier of the cluster, Sessions [ ]: Sessions in thecluster, Centroid: Centroid calculated in different ways, depending onthe case. Each library can provide a function called get_familiarity (asalready mentioned above), that takes as input a dataframe containing theclusters previous calculated and a data frame containing a set of newsessions (each session must have the same shape of the data fame used togenerate the cluster data frame). This function returns a score offamiliarity for each session in the input set. This function does notupdate the clusters and simply assigns each new session to an existingcluster and return a slightly modified session-familiarity of thatcluster (return the session familiarity of the sessions contained inthat cluster, calculated as if the new session were contained in it).The function returns −1 if the session comes from a new user.

In a Familiarity Link Library, e.g. of databricks, a familiarityfunction can be implemented having as input one row for each session andthe following fields: (i) UserID: Identifier for the user, (ii)StartTimeUTC: Start time of the session, used as a session ID, (iii)LinkIDs [ ]: Set of links traveled by the user in the session. Theabsolute value of the LinkID can e.g. be taken in order to consider justthe link and not the travelled direction. Further, in a Familiarity LinkDeployable, e.g. of databricks, an example of the usage of the previouslibrary Familiarity Link Library can be provided. The environment can beselected on the widget and the function saves the three resultsdataframes on the three variables familiarity_user, familiarity_sessionand clusters. This databricks can be deployed on the describedenvironments.

In a Familiarity Start Stop Library, a function can e.g. be providedwhich needs in input a data frame with the following composition: (i)UserID:User ID; (ii) StartTimeUTC: Start time of the session, used as asession ID; (iii) Coordinates{‘lat_a’:StartLatitude,‘long_a’:StartLongitude, ‘lat_b’:EndLatitude, ‘long_b’:EndLongitude}: astructure containing the information of starting and ending points ofthe session. As an example library of the Familiarity Start Stop Librarya Familiarity Start Stop Deployable can e.g. be provided, e.g. asanother databricks. This is an example of the usage of the previouslibrary. The environment can be selected on the widget and the functionsaves the three results dataframes on the three variablesfamiliarity_user, familiarity_session and clusters. This databricks cane.g. be deployed on the described environments.

Further by e.g. a Familiarity Bag of Links, the output data frames canbe different from the previous cases. The three tables can have thefollowing shape: (1) Familiarity_user: (i) UserID: identify the user,and (ii) UserFamiliarity: familiarity of user, calculated as describedabove; (2) Familiarity_session: (i) UserID: identify the user, (ii)SessionID: identify the session, and (iii) SessionFamiliarity:familiarity of session, calculated as described above; (3) Scores: (i)UserID: identify the user, (ii) LinkID: identify the link, and (iii)scores: score of the link, calculated as described above. The scorestable substitutes the cluster table. When it is desired to generate thefamiliarity of a set of new sessions, the get_familiarity of thislibrary can be used but passing the scores data frames, instead of thecluster one. The functions of this library can e.g. be implemented toneed the input with the following shape df: (i) UserID: identify theuser, (ii) StartTimeUTC: starting time of the session, used asSessionID, and (iii) Links[ ]: array containing the absolute values oflinkID of links traveled by the user in the correspondent session. AFamiliarity Bag of Links Deployable can be provided as an example of theusage of the previous library. The environment can be selected on thewidget and the function saves the three results dataframes on the threevariables familiarity_user, familiarity_session and score. Thisdatabricks can be deployed on the des cribbed environments.

Trip Familiarity Detection 114

As an embodiment variant, a trip familiarity detection and measuring 115can be realized as an integrated detection engine based on the abovedescribed Driver Passenger Detection (DPD) 112, Transport ModeRecognition (TMR) 113 and trip familiarity score measuring 114. I.e. thetrip familiarity detection can be realized using TMR 113 measuringsimilarity with annotated trips, DPD 112 measuring familiarity throughthe above described LinkID v1, and the Familiarity Score measuring 114using (i) the familiarity through the described LinkID v2, (ii) start &stop, and (iii) bag of links. A total of 5 different exemplaryfamiliarity clustering data processing and algorithms are disclosedherein. However, other processes are imaginable based on the disclosedtechniques.

First, the disclosed TMR 113 is used providing the inventive technicalstrategy and data considerations. When a TMR 113 request is receivedlive, the system 1 respectively the TMR 113 checks if a user alreadyannotated or corrected a similar trip. Consequently, the system 1 mustbe able to efficiently retrieve historical annotated trip data anddefine a trajectory similarity measure. Since the TMR 113 live requestcontains a representation of the trip with 19 points, in the presentembodiment variant, it makes sense to store this representation for eachannotated trip, partitioned by a user identifier. This can e.g. be donein a database or a filesystem (e.g. one row per trip). The userannotation preferably can e.g. be stored together with the trip summary.This trip summary can be built/updated in batch using, for example,Databricks (e.g. nightly). The embodiment variant can imply informationavailability within 24/48 h from user annotation. Existing facilitiesand other approaches can be considered as well (cf. FIG. 26 ). Forweighting the parameters and evaluating the performance under TMR 113,the multiclass probabilities can e.g. be weighted less than theannotation probability. This is in line with the fact, that, if the usercorrected a trip in the past and a similar trip was observed by thesystem 1, the user should be trusted. The proposed value for the weightis 0.4. FIG. 27 show an exemplary graph, with a TMR baseline.

An exemplary embodiment variant of the DPD 112, which can be used forthe trip familiarity detection 115, and which can e.g. comprise thefollowing technical steps performed by the system 1 and the tripfamiliarity detection and measuring 115, respectively: (1) Collect userhistory, (2) Cluster similar trips, (3) Define centroid trip, (4) Newtrips arrives: seek match with existing clusters, and (5) Check clusterDPD label. This is illustrated by FIG. 28 , where N is the total numberof sessions with DPD score in the cluster, where D_(i)∈[0, 1]P_(i)∈[0,1]and X_(i)∈[0, 1] are final confidence scores returned by DPD for eachsessions (including enter/exit and BT connection), and where clusterscores can be also generated from user annotations (Truth) or eventuallyfrom a combination of both sources.

The objective of the familiarity score is to create a measure forscoring purposes on how much a user travel on familiar roads. This cane.g. require the three different methods, as illustrated by FIG. 29 ,i.e. (1) Clustering through linkID, (ii) bag of links: linkIDsfrequency, and (iii) start & stop. The start&stop method, as illustratedin FIG. 30 is in this context a powerful approach.

DPD used in the context of familiarity detection 115 can comprise thefollowing: (1) For each user: (i) collect trip history (˜few weeks),(ii) cluster similar trips (hierarchical agglomerative clustering viaJaccard distance

${J\left( {A,B} \right)} = \frac{❘{A\bigcap B}❘}{❘{A\bigcup B}❘}$

where trips that share 80% of the geocoding measuring (e.g. HERE) linksare defined similar), and (iii) assign DPD average label to the cluster(using both user annotations+algorithm results); and (2) For new triparriving: (i) seek match with existing clusters (Jaccard distancebetween new trip & the centroids), and (ii) check cluster DPD label.

FIG. 30 shows an exemplary overview of a possible general architectureof the trip familiarity detection and measuring 115. It has to be notedthat to measure similarity between trajectories can be computationallychallenging in regard to the performance and consumption of the system1. Thus, as an embodiment variant, a similarity prefilter can be used inthe system 1, in particular for TMR 113, where the data processing isonly performed on a subset of likely candidates. A trip is considered avalid candidate of its start and end both lie within a certain radiusfrom the start/end of the current trip (the one that is evaluating in aTMR live request). The radius can e.g. be set to 500 meters for thisexample, based on empirical observation. Since user annotations can bein limited number (in normal operating conditions) and using theproposed similarity prefilter, the trajectory similarity is actuallygenerated against a small subset of trips, which is illustrated in FIG.31 .

As a further embodiment variant, e.g. to further improve the performanceof the system 1, a Driver DNA measurement can e.g. be applied andperformed by the system 1. One of the aims of the system 1 and e.g. acorresponding telematics app is to measure and to score the driverbehavior through the recording of GPS, Accelerometer, Gyroscope andother integrated sensors present in personal mobile phone or blackboxes. Different combination of driver and transport mode have differentdriving style, moreover each driver has a different driving styledepending on external factors e.g. weather, road type, and on personalfactors e.g. motivation of the trip, time constraints and tripfamiliarity. Given previous assumptions, the transport mode recognition113 and driver passenger detection 112 can be improved based on an indepth recognition and/or analysis of a single person driving style incombination with his trip history by the system 1. Another aim oftelematics app is the machine-based coaching of the driver to reduce hisrisk while improving his driving style. The analysis of the drivingstyle for each user with a related risk estimation will allow to providepersonalized feedbacks and programs to reduce the risk exposure of eachdriver after a minimum amount of trip history. As an embodiment variant,different assumption for designing features that can contribute totechnically define a driving style, can be used as follows: (i)Correlation between accelerometer and GPS speed, (ii) Frequency ofmaneuvers and phone distraction events per kilometers, (iii) In depthanalysis of speed distribution while turning taking into considerationcurvature degrees, (iv) Analysis of speed distribution taking inconsideration road sinuosity, speed limit and road class, and (v)Analysis and feature extraction from accelerometer and gyroscopedistribution as a function of road class, sinuosity and shape. Usingclustering algorithms together with the above feature extracted from anhistorical set of trips of a single user allows to define and measurethe driver's driving style. For the Driver DNA, as defined above, thesystem 1 clusters the feature measuring and describing the driving styleof a user and to correlate each cluster with the frequency of transportmode, driver or passenger trips present in the cluster. In the end foreach cluster there will be a rank of possible transport mode and a mostprobable output of driver or passenger. This combination is what iscalled herein the measuring of a DriverDNA.

Driver Passenger Detection (DPD) 112

For identifying and/or classifying an occupant of a vehicle 41, 42, 43,. . . based on sensory data measured by a plurality of sensors 102 of acellular mobile device 10 of the occupant 6/61/62, the plurality ofsensors 102 at least comprise an accelerometer 1025 and a gyroscope1026. The mobile device 10 further comprises one or more wirelessconnections 105, wherein by at least one of the wireless connection, thecellular mobile device 10 acts as a wireless node 221, . . . , 225within a cellular data transmission network 2 by means of antennaconnections of the cellular mobile device to the cellular datatransmission network 2, and the plurality of sensors 102 being connectedto a monitoring mobile node application 101 of the mobile device 10. Theone or more wireless connections 105 or wired connections of the mobiletelecommunication apparatus 10 can for example comprise Bluetooth aswireless connection for exchanging data using short-wavelength UHF(Ultra high frequency) radio waves in the ISM (industrial, scientificand medical) radio band from 2.4 to 2.485 GHz by building a personalarea networks (PAN) with the on-board Bluetooth capabilities and/or 3Gand/or 4G and/or GPS and/or Bluetooth LE (Low Energy) and/or BT based onthe Wi-Fi 802.11 standard, and/or a contactless or contact smart card,and/or a SD card (Secure Digital Memory Card) or another interchangeablenon-volatile memory card. For providing the wireless connection 105, themobile telecommunication apparatus 10 can for example act as a wirelessnode within a corresponding data transmission network by means ofantenna connections of the mobile telecommunications apparatuses 10, inparticular, as mentioned, mobile telecommunication networks such as 3G,4G, 5G LTE (Long-Term Evolution) networks or mobile WiMAX or otherGSM/EDGE- and UMTS/HSPA-based network technologies etc., and moreparticularly with appropriate identification means as SIM (SubscriberIdentity Module) etc..

The monitoring mobile node application 101 captures usage-based and/oruser-based telematics data of the cellular mobile device 10 and/or theuser 6/61/62 of the cellular mobile device 10. The mobiletelecommunications apparatuses 10 and the monitoring cellular mobilenode application 101 can e.g. be connected to an on-board diagnosticsystem 431, . . . , 435 and/or an in-car interactive device 441, . . . ,445, wherein the mobile telecommunications apparatuses 10 captureusage-based 31 and/or user-based 32 automotive data 3 of the motorvehicle 41, 42, 43, . . . and/or user. The mobile telecommunicationsapparatuses 10 can for example provide the one or more wirelessconnections 1024 by means of radio data systems (RDS) modules 10241and/or positioning system 10242 including a satellite receiving moduleand/or a mobile cellular phone module 10243 including a digital radioservice module and/or a language unit 10244 in communication with theradio data system 10241 or the positioning system 10242 or the cellulartelephone module 10243. The satellite receiving module 10242 can forexample comprise a Global Positioning System (GPS) circuit and/or thedigital radio service module comprises at least a Global System forMobile Communications (GSM) unit. The plurality of interfaces of themobile telecommunications apparatuses 10 for connection with at leastone of a motor vehicle's data transmission bus can for example compriseat least on interface for connection with a motor vehicle's ControllerArea Network (CAN) bus, e.g. in connection with an on-board diagnostics(OBD) port, or another connection for example for battery installeddevices, or also OEM (Original Equipment Manufacturer) installed systemsobtaining information access to on-board sensors or entertainmentsystems (such as Apple Carplay etc.) providing the necessary vehiclesensor information.

As mentioned, a data link 21 is set by means of the wireless connection105 of the mobile telecommunication apparatus 10 over the mobiletelecommunication network 2 between the mobile telematics application101 as client and an intelligent central automotive circuit 11, whereinthe mobile telecommunication apparatus 10 acts as wireless node 221, . .. , 225 within said mobile telecommunication network 2, and wherein theoperating parameters 40121 and the environmental parameters 40111 aremeasured and collected in dataflow pathway 103 as automotive telematicsdata 3 during operation of the motor vehicle 41, 42, 43, . . . via themobile telecommunication apparatus 10 by means of a mobile telematicsapplication 101 and transmitted to the central circuit 11. Theintelligent central circuit 11 comprises a sensory-data-driven coreaggregator 110 with a plurality of dynamically applied sensorydata-based triggers 1012 triggering, capturing, and monitoring saidsensory parameters in the dataflow pathway 103 by means of a mobiletelematics application 101 of the mobile telecommunication apparatus 10.The mobile telecommunication apparatus 10 can for example comprise atleast a GPS module (Global Positioning System) and/or geological compassmodule based on a 3-axis teslameter and a 3-axis accelerometer, and/orgyrosensor or gyrometer, and/or a MEMS accelerometer sensor comprising acantilever beam with the seismic mass as a proof mass measuring theproper or g-force acceleration, and/or a MEMS magnetometer or amagnetoresistive permalloy sensor or another three-axis magnetometers.

The mobile device 10 measures gravitational acceleration movementsensory data by means of the accelerometer based on measuring parametersobtained from the accelerometer. Vehicle 41, 42, . . . entering orexiting movement patterns of the user are detected from the accelerationmovement sensory data at least comprising pattern for base axis anddegree of rotation associated with a vehicle entrance or exit of theuser 6. The detected vehicle entering or exiting movement patterns ofthe user 10 trigger as input features the recognition of a vehicleentering or exiting movement of the user by performing a decision-treeclassification on the input features to rule out whether the userentered or exited from a left or right side of the vehicle. It is to benoted that the system 1 can also be realized by using otherclassification algorithms or structures e.g. boosted tree or neuralnetwork etc.

The DPD system 112 allows to select (as few as possible) characteristicinput features to reduce the number of model parameters to be used. Theinventive DPD (Driver Passenger Detection) method and system comprise atleast the following three main steps: 1. Detect the exact moment whenthe user is entering/exiting the car by analyzing the acceleration. 2.Use the gyroscope data to select various features such as the verse andthe degree of the rotation associated to the entrance/exit. 3. Perform adecision-tree classification on the input features to rule out whetherthe user entered (exited) from the left/right side of the car. Thesystem provides a detection of the exact moment when a person isentering/exiting the car. It is to be mentioned that without thisinformation, any other analysis of the Gyroscope sensor will be uselessto the DPD problem due to the many rotations that a user can perform ina huge variety of movements. The detection step is accomplished bycollecting information both on the variance of the acceleration in theup/down (Earth reference system) directions and on the presence (or not)of some particular discontinuities in the acceleration signals in thesmartphone reference system (not rotated).

One of the advantages of the present invention is its easy adaptabilityand suitability for its use in modular systems, e.g. to technicallyprovide familiarity detection of trips. Thus, the present DriverPassenger Detection (DPD) system can e.g. be realized as part of aninventive, more complex, and composite modular monitoring and detectionsystem 1 with interactive Driver Passenger Detection (DPD) 112,Transport Mode Recognition (TMR) 113 and trip familiarity detectionand/or score 114, allowing a broad monitoring of user actions related tothe use of his/her mobile phone.

LIST OF REFERENCE SIGNS

1 Mobile identification and classification system

-   -   10 Mobile telecommunications apparatus        -   101 Mobile telematics application (cellular mobile node            application)        -   102 Integrated Sensors of the mobile node            -   1020 MEMS magnetometer            -   1021 Proximity Sensor            -   1022 Fingerprint Sensor            -   1023 Ambient Light Sensor            -   1024 GPS Sensor                -   10241 Longitude position                -   10242 Latitude position                -   10243 Altitude position            -   1025 Accelerometer            -   1026 Gyroscope            -   1027 Cameras            -   1028 Touchscreen            -   1029 MEMS compass module            -   1030 Back Illuminated Sensor            -   1031 NFC Sensor        -   103 Dataflow pathway        -   105 Wireless connections            -   1051 GSM            -   1052 WLAN            -   1053 Bluetooth            -   1054 Near Field Communication NFC (for NFC Sensors)    -   11 Central circuit        -   110 Telematics-driven aggregator            -   1101 Data Interface        -   111 Machine-learning module        -   112 Driver Passenger Detection (DPD) system        -   113 Transport Mode Recognition (TMR)            -   1131 Gradient boosting machine-learning classifier            -   1132 Input feature values            -   1133 Transportation modes                -   11331 Public transportation                -   11332 Motorcycle                -   11333 Cycling                -   11334 Train                -   11335 Tram                -   11336 Plane                -   11337 Car                -   11338 Skiing                -   11339 Boat            -   1134 Transport mode label (Output value)            -   1135 Trips                -   11351 Transport mode movement pattern            -   1136 Supervised learning structure        -   114 Trip familiarity Measuring and Detection    -   12 First-tier automated risk-transfer system        -   121 Electronic first-tier resource-pooling system        -   122 First-tier risk-transfer parameters        -   123 First-tier payment-transfer parameters    -   13 Second-tier automated risk-transfer system        -   131 Electronic second-tier resource-pooling system        -   132 Second-tier risk-transfer parameters        -   133 Second-tier payment-transfer parameters

2 Data transmission network

-   -   20 Cellular network grid        -   201, . . . , 203 Network cell/Basic service area        -   211, . . . , 213 Base (transceiver) station            -   2111, . . . , 2131 Cell Global Identity (CGI)        -   221, . . . , 226 Cellular network node    -   21 Uni- or bidirectional data link

3 Sensory data of the mobile device 10

-   -   31 Sensory parameter values of the 3-axis accelerometer    -   32 Sensory parameter values of the GPS sensor    -   33 Trips database        -   331 Measured Time Serie of Sensory Parameter Values of            Stored Trips 1, . . . , t

41, 42,43, . . . Motor vehicles

-   -   401, . . . , 405 On-board sensors and measuring devices    -   411, . . . , 415 OEM (Original Equipment Manufacturer) devices    -   421, . . . , 425 Data transmission bus interface    -   431, . . . , 435 On-board diagnostic system    -   441, . . . , 445 In-car interactive device    -   451, . . . , 455 Automotive telematics devices

6 User of the mobile device

1. A method for automated transportation mode recognition based onsensory data measured by a plurality of sensors of a mobile device of auser, the plurality of sensors at least comprising an accelerometer anda GPS sensor, the mobile device comprising one or more wirelessconnections, the mobile device acting as a wireless node within acellular data transmission network by means of antenna connections ofthe mobile device to the cellular data transmission network, theplurality of sensors being connected to a monitoring mobile nodeapplication of the mobile device, and the monitoring mobile nodeapplication capturing usage-based and/or user-based sensory data of themobile device and/or the user of the mobile device, the methodcomprising: measuring time series of sensory parameter values based onmeasuring parameters obtained from the plurality of sensor of the mobiledevice, the measuring parameters comprising a time series of sensoryparameter values of the accelerometer measurements and a time series ofsensory parameter values of GPS-based speed measurements of the GPSsensor, the GPS sensor measuring longitude, latitude, and altitudepositions of the mobile device by measuring different speeds of lightdelays in signals coming from two or more satellites, and triggering theautomated transportation mode recognition using the measured time seriesof the sensory parameter values as input feature values to a gradientboosting machine-learning classifier, wherein the transportation modeincludes at least one of public transportation, motorcycle, cycling,train, tram, plane, car, skiing, and boat, and the transportation moderecognition generates a transport mode label for a transport modemovement pattern of a trip.
 2. The method for automated transportationmode recognition according to claim 1, wherein a supervised learningstructure is applied to the gradient boosting machine-learningclassifier during a supervised learning phase, transport mode movementpatterns of measured trips are stored in a trips database, the sensoryparameter values include sensory movement parameter values, transportmode movement patterns of the trip are identified from the sensorymovement parameter values, each of the trips comprises the sensorymovement parameter values of GPS positions by the GPSsensor andacceleration forces being applied to the mobile device on all threephysical axes by the accelerometer, operating system activitiesparameter values of an operating system of the mobile device, and atransport mode label value, and trips with transport mode labelsdetected by the gradient boosting machine-learning classifier are fedinto a user back-loop for dynamic correction by a user associated with arespective trip and saved to the trips database by updating learningtransport mode movement patterns of the measured trips in the tripsdatabase.
 3. The method for automated transportation mode recognitionaccording to claim 2, further comprising monitoring the trips databaseto automatically detect changes in the trips database, wherein upondetecting a change in the trips database, the supervised learning phaseis reinitiated.
 4. The method for automated transportation moderecognition according to claim 2, wherein the trips database is updatedcontinuously and/or dynamically based on the user back-loop.
 5. Themethod for automated transportation mode recognition according to claim2, wherein, for the supervised learning phase, the trips of the tripsdatabase are preprocessed by filtering out transport mode movementpatterns, which have a time duration shorter than 1 minute and/orcomprise less than 30 GPS positions and/or do not have a propertransport mode labelling.
 6. The method for automated transportationmode recognition according to claim 2, wherein, for the supervisedlearning phase, the trips of the trips database are preprocessed byfiltering out transport mode movement patterns having duplicated GPSlocations by timestamp and/or transport mode movement patterns with GPSlocations having negative speed and/or transport mode movement patternshaving GPS locations with an accuracy>50 m.
 7. The method for automatedtransportation mode recognition according to claim 1, wherein, upondetection by the gradient boosting machine-learning classifier and thegeneration of the transport mode label for the transport mode movementpattern of the trip, the trips are postprocessed by a set of hard codedrules, reinforcing avoidance of incorrect and/or insufficientlyconfident recognition.
 8. The method for automated transportation moderecognition according to claim 1, wherein users routines areautomatically detected by a trip familiarity-based recognition structureincreasing the accuracy of the automated transportation moderecognition.
 9. The method for automated transportation mode recognitionaccording to claim 2, wherein the transport mode movement patterns ofmeasured trips stored to the trips database are processed for dataenrichment, where the data enrichment process comprises route-matchingof the trips with the transport mode movement patterns based on roadmapsand/or GIS-geometry mapping of the trips with the transport modemovement patterns based on spatial and geographic GIS data, and/orpublic transport mapping based on public transport road maps andtimetable data.
 10. The method for automated transportation moderecognition according to claim 1, wherein the measuring parameters ofthe time series of sensory parameter values of the GPS-based speedmeasurements further comprise extracted average GPS-based speedmeasurements and/or standard deviation of the GPS-based speedmeasurements and/or percentiles values from 0 to 100 of the GPS-basedspeed measurements in predefined percentile steps.
 11. The method forautomated transportation mode recognition according to claim 10, whereinthe percentile steps comprise a granularity of a factor
 10. 12. Themethod for automated transportation mode recognition according to claim1, wherein the altitude position further comprises a standard deviationextracted from the altitude position measurements.
 13. The method forautomated transportation mode recognition according to claim 1, whereinthe measured time series of the sensory parameter values compriseGPS-based acceleration measurements derived based on a measured ratiobetween (a) a measured speed difference between a measured set of GPSparameter values and a measured subsequent set of GPS parameter values,and (b) a measured time difference between the measured set of GPSparameter values and the measured subsequent set of GPS parametervalues.
 14. The method for automated transportation mode recognitionaccording to claim 13, wherein the GPS-based acceleration measurementscomprise a standard deviation extracted from the GPS-based accelerationmeasurements and/or a variance value of the GPS-based accelerationmeasurements derived based on an angle between triplets of consecutiveGPS points.
 15. The method for automated transportation mode recognitionaccording to claim 1, wherein the measuring parameters of the timeseries of sensory parameter values of the accelerometer measurementsfurther comprise percentile values from 0 to 100 of the accelerometermeasurements in predefined percentile steps and/or interquartile rangevalues measured by a difference between the 75^(th) and the 25^(th)percentile.
 16. The method for automated transportation mode recognitionaccording to claim 15, wherein said percentile steps comprise agranularity of a factor
 10. 17. The method for automated transportationmode recognition according to claim 2, wherein in a case of two or moreaccelerometer measurements having the same timestamps, a last one withrespect to an accelerometer measurement order is selected, and anaccelerometer measurement norm value is generated over the accelerometermeasurements of a measured trip, and an average of the accelerometermeasurements of the measured trip is removed form said measured trip.18. The method for automated transportation mode recognition accordingto claim 2, wherein the operating system activities parameter values ofthe operating system of the mobile device comprise a unique timestampand a map of labels with probability measures, and in a case of anabsence of a label, the probability measure is set to
 0. 19. The methodfor automated transportation mode recognition according to claim 18,wherein said labels of the map are normalized to ‘Automotive’,‘Cycling’, ‘OnFoot’, ‘Running’, ‘Stationary’, ‘Unknown’, ‘Walking’, and‘Tilting’ denoting a feature vector for naming compliance between twooperating systems.
 20. The method for automated transportation moderecognition according to claim 18, further comprising assuming a labelprobability is valid until a next event is measured, wherein each labelprobability is multiplied by measured milliseconds elapsed until thenext event, or until an end of the trip for a last received activityevent, the multiplication is performed for each label of a label list,the multiplied label probabilities for each label are summed up, andeach sum is divided by a difference between a trip end time and a firstactivity event time, both in milliseconds.
 21. The method for automatedtransportation mode recognition according to claim 20, wherein, in acase of a label being never returned, a corresponding feature is set to0, so that if there are no activities at all for a measured trip, allthe operating system activities parameter values are set to
 0. 22. Themethod for automated transportation mode recognition according to claim9, wherein the public transport mapping based on public transport roadmaps and timetable data comprises identifying for a set of measured GPSlocation parameters candidate stops as sequences of points that fulfillconditions of: (i) measured speed<=3 m/s; and (ii) candidate sequencesare longer than 5 seconds, the identification is performed afterapplying a moving average with window length 9 over an array of measuredspeeds, replacing each sample an average of the sample itself and the 4samples before and after, and for each of the candidate sequences, anaverage latitude and an average longitude is generated, obtaining acandidate stop position for each sequence/stop.
 23. The method forautomated transportation mode recognition according to claim 1, whereina hyperparameter configuration is applied to the gradient boostingmachine-learning classifier comprising the values 225 for then-estimators, 0.03 for the learning-rate, 30 for the max-depth, 50 forthe num-leaves, 0.8 for the subsample, 0.7 for the colsample-bytree, and5 for the min-sum-hessian-in-leaf.